The tokenizer (the code that breaks up the line given to the shell into
strings to create argv) was quite a messy piece of code. This commit
refactors it into a more traditional state-machine based parser.
This fixes the issues with quote handling exposed by the recently
introduced test.
Co-authored-by: Juan Carrano <j.carrano@fu-berlin.de>
86f60357cf