Saving frequencies as Dictionaries may be faster than Bags

[olekscode]

Profile NgramModel with frequencies saved as Dictionary and compare it to the current one that saves frequencies in Bag

[jecisc]

I did not check a lot the implementation but maybe we could also have a tree and each leave would be a letter. Thus, when checking the counts of "test" we would go in node "t" then "e" then "s" then "t" and it would be faster than scanning a dictonary with hundreds of thousands entries?

Once again, I don’t know this algo and just had a quick look, so maybe my solution does not fit the requirements.