Tokenization
White space tokenization only separates sentences into words. It didn’t separate compound or contraction words so it kept the meaning of those words consistent. I don’t think this method is enough for effectively tokenizing the English language due to contractions having different meanings in different contexts and that phrases work in the same way. Whitespace tokenization would need to be used in tandem with a phrase tokenizer to pick up more meaningful and effective patterns.
Trying a CALL Tool
I tried mango languages for this assignment. The free lessons that it supplies don’t actually give any feedback that I could see. It provides things to compare your pronunciation to, but it doesn’t do anything specifically with the data that it collects from you. The free lessons don’t get into actually answering questions. You are able to quickly click through the entire lesson and the site gives you credit for completing the lesson. Clozes don’t exist here because answer choices or verbal input aren’t evaluated by the website. The lessons focus on grammar and conjugation foundations of sentences while there is another separate tab for vocabulary that goes over what words were used and learned in the lessons.