Comments on: Code: Tokenizer http://jet.ro/2009/11/10/code-tokenizer/ designer/programmer tidbits Tue, 20 Dec 2011 03:09:51 +0000 hourly 1 http://wordpress.org/?v=3.2.1 By: jetro http://jet.ro/2009/11/10/code-tokenizer/comment-page-1/#comment-71 jetro Sun, 24 Jan 2010 22:07:59 +0000 http://jet.ro/?p=782#comment-71 Thanks for the tip, I updated the zip contents. Thanks for the tip, I updated the zip contents.

]]> By: Emil http://jet.ro/2009/11/10/code-tokenizer/comment-page-1/#comment-69 Emil Sat, 23 Jan 2010 20:29:05 +0000 http://jet.ro/?p=782#comment-69 You may want to change lines 111 and 112 of Tokenizer.cpp from: if (str[a] == separators[b]) ++mCount; to: if (str[a] == separators[b]) { ++mCount; break; } You may want to change lines 111 and 112 of Tokenizer.cpp from:

if (str[a] == separators[b])
++mCount;

to:

if (str[a] == separators[b]) {
++mCount;
break;
}

]]>