# lexical demand report # For "C:\CANADA-@25\CA501" by on Thu Aug 25 09:49:45 2005 # ID code is &&111 Lexicon "english" contains 32159 words. right boxes ABSOLUTE area = 111.79 #Word listing for cumulative proportion distribution from 1 to 32159: # NOTE: Percentages are relative to the number of legitimate tokens. #RANK CORPUS% CUM.% WIDTH CORP.AREA N SAMPLE% DIFF% CUM.% RATIO AREA NET WORD 1 6.15914 6.1591 0.301030 1.8541 42 5.76923 -0.38991 5.77 0.94 1.7367 0.117373 %the 2 2.72130 8.8804 0.176091 1.5638 18 2.47253 -0.24877 8.24 0.91 1.4513 0.112466 %to 3 2.59284 11.4733 0.124939 1.4335 0 0.00000 -2.59284 8.24 0.00 1.0297 0.403742 %of 4 2.55127 14.0246 0.096910 1.3591 11 1.51099 -1.04028 9.75 0.59 0.9451 0.413980 %a 5 2.38627 16.4108 0.079181 1.2994 27 3.70879 1.32252 13.46 1.55 1.0659 0.233528 %and 6 2.10793 18.5188 0.066947 1.2398 8 1.09890 -1.00903 14.56 0.52 0.9748 0.264996 %in 7 1.15113 19.6699 0.057992 1.1407 5 0.68681 -0.46432 15.25 0.60 0.8842 0.256477 %that 8 1.00620 20.6761 0.051153 1.0576 7 0.96154 -0.04466 16.21 0.96 0.8291 0.228513 %for 9 0.91579 21.5919 0.045757 0.9880 17 2.33516 1.41938 18.54 2.55 0.8485 0.139465 %is 10 0.85364 22.4455 0.041393 0.9291 33 4.53297 3.67933 23.08 5.31 0.9552 -0.026136 %said 12: 0.71655 23.9135 0.034762 0.8313 19 2.60989 1.89334 25.69 3.64 0.8929 -0.061644 %he 13 0.69811 24.6116 0.032185 0.7921 3 0.41209 -0.28603 26.10 0.59 0.8400 -0.047868 %it 14 0.68409 25.2957 0.029963 0.7579 5 0.68681 0.00272 26.79 1.00 0.8026 -0.044645 %with 15 0.66133 25.9570 0.028029 0.7275 2 0.27473 -0.38660 27.06 0.42 0.7585 -0.030927 %was 17: 0.59474 27.1693 0.024824 0.6744 18 2.47253 1.87778 29.53 4.16 0.7331 -0.058674 %at 18 0.49361 27.6629 0.023481 0.6496 10 1.37363 0.88001 30.91 2.78 0.7257 -0.076165 %by 19 0.48297 28.1459 0.022276 0.6270 3 0.41209 -0.07088 31.32 0.85 0.6977 -0.070678 %his 20 0.47241 28.6183 0.021189 0.6064 0 0.00000 -0.47241 31.32 0.00 0.6636 -0.057219 %be 21 0.46512 29.0834 0.020203 0.5876 6 0.82418 0.35905 32.14 1.77 0.6494 -0.061811 %but 23: 0.44987 29.9965 0.018483 0.5544 6 0.82418 0.37431 32.97 1.83 0.6093 -0.054905 %have 24 0.43223 30.4287 0.017729 0.5395 2 0.27473 -0.15750 33.24 0.64 0.5893 -0.049872 %are 25 0.40302 0.4030 0.017033 0.0069 27 3.70879 3.30578 3.71 9.20 0.0632 -0.056308 %i 28: 0.37692 1.5656 0.015240 0.0239 15 2.06044 1.68352 5.77 5.47 0.0879 -0.064063 not 31: 0.34812 2.6319 0.013788 0.0363 6 0.82418 0.47605 6.59 2.37 0.0909 -0.054622 this 32 0.31459 2.9465 0.013364 0.0394 6 0.82418 0.50958 7.42 2.62 0.0991 -0.059751 will 34: 0.28631 3.5239 0.012589 0.0444 3 0.41209 0.12578 7.83 1.44 0.0986 -0.054206 had 35 0.26799 3.7919 0.012234 0.0464 0 0.00000 -0.26799 7.83 0.00 0.0958 -0.049400 their 38: 0.25360 4.5764 0.011281 0.0516 9 1.23626 0.98266 9.07 4.87 0.1023 -0.050647 one 40: 0.24966 5.0769 0.010724 0.0544 3 0.41209 0.16243 9.48 1.65 0.1016 -0.047197 new 43: 0.23767 5.8038 0.009984 0.0579 23 3.15934 2.92167 12.64 13.29 0.1262 -0.068228 you 50: 0.20109 7.3294 0.008600 0.0630 0 0.00000 -0.20109 12.64 0.00 0.1087 -0.045649 if 52: 0.19193 7.7178 0.008273 0.0638 5 0.68681 0.49488 13.32 3.58 0.1102 -0.046379 all 53 0.19160 7.9094 0.008118 0.0642 1 0.13736 -0.05423 13.46 0.72 0.1093 -0.045072 out 56: 0.17919 8.4526 0.007687 0.0650 12 1.64835 1.46916 15.11 9.20 0.1161 -0.051173 she 60: 0.16582 9.1359 0.007179 0.0656 0 0.00000 -0.16582 15.11 0.00 0.1085 -0.042885 no 61 0.16318 9.2991 0.007062 0.0657 3 0.41209 0.24890 15.52 2.53 0.1096 -0.043945 her 62 0.16295 9.4620 0.006949 0.0658 2 0.27473 0.11178 15.80 1.69 0.1098 -0.044019 two 63 0.16179 9.6238 0.006839 0.0658 4 0.54945 0.38766 16.35 3.40 0.1118 -0.045977 can 64 0.16155 9.7854 0.006733 0.0659 4 0.54945 0.38790 16.90 3.40 0.1138 -0.047876 what 66: 0.15713 10.1027 0.006531 0.0660 8 1.09890 0.94177 17.99 6.99 0.1175 -0.051540 like 68: 0.15637 10.4157 0.006340 0.0660 4 0.54945 0.39309 18.54 3.51 0.1176 -0.051534 so 73: 0.14478 11.1609 0.005909 0.0659 1 0.13736 -0.00742 18.68 0.95 0.1104 -0.044437 into 75: 0.14257 11.4478 0.005752 0.0659 0 0.00000 -0.14257 18.68 0.00 0.1075 -0.041610 million 77: 0.13142 11.7109 0.005604 0.0656 4 0.54945 0.41803 19.23 4.18 0.1078 -0.042140 just 80: 0.12567 12.0951 0.005395 0.0653 4 0.54945 0.42378 19.78 4.37 0.1067 -0.041462 now 85: 0.12246 12.7167 0.005080 0.0646 6 0.82418 0.70171 20.60 6.73 0.1047 -0.040066 do 92: 0.09389 13.4221 0.004695 0.0630 15 2.06044 1.96655 22.66 21.94 0.1064 -0.043396 my 93 0.09378 13.5159 0.004645 0.0628 3 0.41209 0.31831 23.08 4.39 0.1072 -0.044410 may 96: 0.08894 13.7851 0.004501 0.0620 1 0.13736 0.04842 23.21 1.54 0.1045 -0.042436 three 98: 0.08773 13.9607 0.004409 0.0616 7 0.96154 0.87381 24.18 10.96 0.1066 -0.045040 then 100: 0.08695 14.1347 0.004321 0.0611 0 0.00000 -0.08695 24.18 0.00 0.1045 -0.043392 where 114: 0.08030 15.3028 0.003793 0.0580 3 0.41209 0.33179 24.59 5.13 0.0933 -0.035218 say 115 0.07768 15.3805 0.003760 0.0578 5 0.68681 0.60913 25.27 8.84 0.0950 -0.037204 did 116 0.07754 15.4580 0.003728 0.0576 1 0.13736 0.05982 25.41 1.77 0.0947 -0.037108 good 129: 0.06953 16.4117 0.003354 0.0550 6 0.82418 0.75465 26.24 11.85 0.0880 -0.032948 me 137: 0.06581 16.9471 0.003159 0.0535 8 1.09890 1.03309 27.34 16.70 0.0863 -0.032811 here 140: 0.06543 17.1438 0.003091 0.0530 2 0.27473 0.20930 27.61 4.20 0.0853 -0.032352 house 141 0.06538 17.2091 0.003069 0.0528 2 0.27473 0.20934 27.88 4.20 0.0856 -0.032765 go 145: 0.06424 17.4676 0.002985 0.0521 5 0.68681 0.62257 28.57 10.69 0.0853 -0.033143 home 148: 0.06207 17.6558 0.002925 0.0516 2 0.27473 0.21265 28.85 4.43 0.0844 -0.032727 city 150: 0.06133 17.7787 0.002886 0.0513 2 0.27473 0.21340 29.12 4.48 0.0840 -0.032730 see 151 0.06056 17.8392 0.002867 0.0511 5 0.68681 0.62625 29.81 11.34 0.0854 -0.034309 too 154: 0.05997 18.0201 0.002811 0.0507 9 1.23626 1.17629 31.04 20.61 0.0873 -0.036610 want 164: 0.05657 18.5987 0.002640 0.0491 4 0.54945 0.49288 31.59 9.71 0.0834 -0.034307 little 175: 0.05263 19.1928 0.002475 0.0475 3 0.41209 0.35946 32.01 7.83 0.0792 -0.031706 got 178: 0.05211 19.3497 0.002433 0.0471 4 0.54945 0.49734 32.55 10.54 0.0792 -0.032129 big 181: 0.05091 19.5029 0.002393 0.0467 3 0.41209 0.36118 32.97 8.09 0.0789 -0.032217 play 183: 0.05076 19.6045 0.002367 0.0464 8 1.09890 1.04814 34.07 21.65 0.0806 -0.034226 come 198: 0.04738 20.3431 0.002188 0.0445 3 0.41209 0.36471 34.48 8.70 0.0754 -0.030926 best 200: 0.04687 20.4370 0.002166 0.0443 0 0.00000 -0.04687 34.48 0.00 0.0747 -0.030414 fell 208: 0.04572 20.8073 0.002083 0.0433 1 0.13736 0.09164 34.62 3.00 0.0721 -0.028762 days 221: 0.04303 21.3805 0.001961 0.0419 1 0.13736 0.09433 34.75 3.19 0.0681 -0.026219 came 222 0.04271 21.4232 0.001952 0.0418 2 0.27473 0.23202 35.03 6.43 0.0684 -0.026554 white 237: 0.03989 22.0388 0.001829 0.0403 8 1.09890 1.05901 36.13 27.55 0.0661 -0.025761 something 250: 0.03854 22.5475 0.001734 0.0391 0 0.00000 -0.03854 36.13 0.00 0.0626 -0.023542 past 256: 0.03782 22.7753 0.001693 0.0386 2 0.27473 0.23690 36.40 7.26 0.0616 -0.023071 run 260: 0.03738 22.9256 0.001667 0.0382 5 0.68681 0.64943 37.09 18.37 0.0618 -0.023611 man 270: 0.03617 23.2943 0.001606 0.0374 7 0.96154 0.92536 38.05 26.58 0.0611 -0.023690 look 272: 0.03604 23.3664 0.001594 0.0372 4 0.54945 0.51341 38.60 15.24 0.0615 -0.024277 away 288: 0.03421 23.9246 0.001505 0.0360 4 0.54945 0.51524 39.15 16.06 0.0589 -0.022917 again 291: 0.03391 24.0267 0.001490 0.0358 5 0.68681 0.65290 39.84 20.25 0.0593 -0.023552 went 300: 0.03295 24.3270 0.001445 0.0352 0 0.00000 -0.03295 39.84 0.00 0.0576 -0.022413 give 314: 0.03226 24.7844 0.001381 0.0342 5 0.68681 0.65456 40.52 21.29 0.0560 -0.021732 old 325: 0.03151 25.1355 0.001334 0.0335 3 0.41209 0.38058 40.93 13.08 0.0546 -0.021079 find 370: 0.02844 26.4890 0.001172 0.0310 2 0.27473 0.24629 41.21 9.66 0.0483 -0.017254 open 400: 0.02636 27.3043 0.001084 0.0296 0 0.00000 -0.02636 41.21 0.00 0.0447 -0.015078 kind 412: 0.02564 27.6142 0.001053 0.0291 1 0.13736 0.11172 41.35 5.36 0.0435 -0.014457 wanted 449: 0.02366 28.5260 0.000966 0.0276 4 0.54945 0.52579 41.90 23.22 0.0405 -0.012917 name 500: 0.02165 29.6848 0.000868 0.0258 0 0.00000 -0.02165 41.90 0.00 0.0364 -0.010596 coming 522: 0.02091 30.1525 0.000831 0.0251 2 0.27473 0.25382 42.17 13.14 0.0351 -0.009989 comes 536: 0.02029 30.4404 0.000809 0.0246 2 0.27473 0.25444 42.45 13.54 0.0344 -0.009718 gave 600: 0.01840 31.6787 0.000723 0.0229 0 0.00000 -0.01840 42.45 0.00 0.0307 -0.007786 republicans 608: 0.01822 31.8251 0.000714 0.0227 3 0.41209 0.39387 42.86 22.62 0.0306 -0.007874 ball 623: 0.01792 32.0965 0.000697 0.0224 2 0.27473 0.25681 43.13 15.33 0.0300 -0.007687 red 662: 0.01696 32.7743 0.000656 0.0215 1 0.13736 0.12040 43.27 8.10 0.0284 -0.006880 morning 667: 0.01689 32.8588 0.000651 0.0214 2 0.27473 0.25784 43.54 16.27 0.0283 -0.006952 mother 669: 0.01687 32.8925 0.000649 0.0213 5 0.68681 0.66995 44.23 40.72 0.0287 -0.007355 saw 699: 0.01618 33.3877 0.000621 0.0207 2 0.27473 0.25855 44.51 16.98 0.0276 -0.006903 brown 700 0.01613 33.4038 0.000620 0.0207 0 0.00000 -0.01613 44.51 0.00 0.0276 -0.006883 social 800: 0.01437 34.9267 0.000543 0.0189 0 0.00000 -0.01437 44.51 0.00 0.0241 -0.005197 showed 821: 0.01409 35.2262 0.000529 0.0186 5 0.68681 0.67273 45.19 48.76 0.0239 -0.005269 pretty 838: 0.01384 35.4635 0.000518 0.0184 1 0.13736 0.12353 45.33 9.93 0.0235 -0.005110 stay 853: 0.01353 35.6690 0.000509 0.0181 2 0.27473 0.26119 45.60 20.30 0.0232 -0.005056 green 900: 0.01288 36.2885 0.000482 0.0175 0 0.00000 -0.01288 45.60 0.00 0.0220 -0.004493 situation 942: 0.01238 36.8189 0.000461 0.0170 1 0.13736 0.12498 45.74 11.10 0.0211 -0.004112 am 1000: 0.01169 37.5145 0.000434 0.0163 20 2.74725 2.73556 48.49 235.05 0.0210 -0.004764 looked 1053: 0.01105 38.1145 0.000412 0.0157 4 0.54945 0.53840 49.04 49.72 0.0202 -0.004503 store 1089: 0.01059 38.5034 0.000399 0.0153 2 0.27473 0.26413 49.31 25.93 0.0197 -0.004309 blue 1177: 0.00981 39.3978 0.000369 0.0145 5 0.68681 0.67700 50.00 69.99 0.0184 -0.003910 happy 1200: 0.00955 39.6205 0.000362 0.0143 0 0.00000 -0.00955 50.00 0.00 0.0181 -0.003755 version 1202: 0.00954 39.6396 0.000361 0.0143 10 1.37363 1.36408 51.37 143.93 0.0186 -0.004238 ran 1400: 0.00824 41.3869 0.000310 0.0128 0 0.00000 -0.00824 51.37 0.00 0.0159 -0.003097 learn 1418: 0.00813 41.5343 0.000306 0.0127 3 0.41209 0.40395 51.79 50.66 0.0159 -0.003139 box 1503: 0.00775 42.2090 0.000289 0.0122 3 0.41209 0.40434 52.20 53.19 0.0151 -0.002885 yes 1515: 0.00769 42.3016 0.000287 0.0121 2 0.27473 0.26703 52.47 35.72 0.0150 -0.002915 walk 1564: 0.00746 42.6711 0.000278 0.0118 2 0.27473 0.26727 52.75 36.83 0.0146 -0.002797 stopped 1595: 0.00727 42.8996 0.000272 0.0117 7 0.96154 0.95427 53.71 132.24 0.0146 -0.002942 mr 1600: 0.00723 42.9359 0.000271 0.0117 0 0.00000 -0.00723 53.71 0.00 0.0146 -0.002923 block 1698: 0.00672 43.6174 0.000256 0.0112 3 0.41209 0.40536 54.12 61.29 0.0138 -0.002686 garden 1753: 0.00652 43.9814 0.000248 0.0109 2 0.27473 0.26821 54.40 42.14 0.0135 -0.002579 girl 1800: 0.00632 44.2835 0.000241 0.0107 0 0.00000 -0.00632 54.40 0.00 0.0131 -0.002439 loan 1805: 0.00629 44.3150 0.000241 0.0107 1 0.13736 0.13107 54.53 21.83 0.0131 -0.002458 walked 1820: 0.00620 44.4087 0.000239 0.0106 4 0.54945 0.54325 55.08 88.61 0.0131 -0.002546 surprise 2000: 0.00561 45.4698 0.000217 0.0099 0 0.00000 -0.00561 55.08 0.00 0.0120 -0.002087 uses 2050: 0.00547 45.7465 0.000212 0.0097 4 0.54945 0.54398 55.63 100.39 0.0118 -0.002094 train 2176: 0.00517 46.4198 0.000200 0.0093 2 0.27473 0.26955 55.91 53.10 0.0112 -0.001893 ride 2247: 0.00500 46.7813 0.000193 0.0090 5 0.68681 0.68181 56.59 137.25 0.0109 -0.001896 dog 2277: 0.00493 46.9304 0.000191 0.0089 9 1.23626 1.23133 57.83 250.61 0.0110 -0.002078 eat 2414: 0.00459 47.5826 0.000180 0.0086 4 0.54945 0.54486 58.38 119.78 0.0105 -0.001942 oh 2500: 0.00441 47.9688 0.000174 0.0083 0 0.00000 -0.00441 58.38 0.00 0.0101 -0.001808 =olympics 2502: 0.00440 47.9776 0.000174 0.0083 1 0.13736 0.13296 58.52 31.20 0.0102 -0.001829 animals 2521: 0.00436 48.0607 0.000172 0.0083 2 0.27473 0.27037 58.79 63.05 0.0101 -0.001848 window 2547: 0.00432 48.1734 0.000170 0.0082 1 0.13736 0.13305 58.93 31.83 0.0100 -0.001834 likes 2574: 0.00426 48.2892 0.000169 0.0081 3 0.41209 0.40782 59.34 96.62 0.0100 -0.001864 wish 2940: 0.00363 49.7218 0.000148 0.0073 5 0.68681 0.68319 60.03 189.31 0.0089 -0.001522 boat 2959: 0.00360 49.7904 0.000147 0.0073 1 0.13736 0.13376 60.16 38.15 0.0088 -0.001522 liked 3000: 0.00353 49.9368 0.000145 0.0072 0 0.00000 -0.00353 60.16 0.00 0.0087 -0.001480 =sam 3500: 0.00290 51.5329 0.000124 0.0064 0 0.00000 -0.00290 60.16 0.00 0.0075 -0.001071 reasonable 3596: 0.00281 51.8073 0.000121 0.0063 5 0.68681 0.68400 60.85 244.16 0.0073 -0.001092 bird 3934: 0.00251 52.7055 0.000110 0.0058 3 0.41209 0.40957 61.26 163.92 0.0068 -0.000945 breakfast 4000: 0.00246 52.8696 0.000109 0.0057 0 0.00000 -0.00246 61.26 0.00 0.0067 -0.000911 rushing 4132: 0.00235 53.1873 0.000105 0.0056 7 0.96154 0.95919 62.23 408.99 0.0065 -0.000950 birds 4333: 0.00221 53.6456 0.000100 0.0054 5 0.68681 0.68460 62.91 310.78 0.0063 -0.000929 toy 4500: 0.00211 54.0055 0.000096 0.0052 0 0.00000 -0.00211 62.91 0.00 0.0061 -0.000859 resignation 4630: 0.00203 54.2738 0.000094 0.0051 3 0.41209 0.41006 63.32 203.30 0.0059 -0.000849 gate 5000: 0.00182 54.9860 0.000087 0.0048 0 0.00000 -0.00182 63.32 0.00 0.0055 -0.000724 defendants 5500: 0.00160 55.8425 0.000079 0.0044 0 0.00000 -0.00160 63.32 0.00 0.0050 -0.000591 potatoes 5792: 0.00148 56.2935 0.000075 0.0042 7 0.96154 0.96006 64.29 649.69 0.0048 -0.000599 coat 6000: 0.00141 56.5941 0.000072 0.0041 0 0.00000 -0.00141 64.29 0.00 0.0047 -0.000557 midst 6177: 0.00136 56.8392 0.000070 0.0040 5 0.68681 0.68545 64.97 505.01 0.0046 -0.000572 laughed 6500: 0.00125 57.2620 0.000067 0.0038 0 0.00000 -0.00125 64.97 0.00 0.0043 -0.000515 stepping 6566: 0.00123 57.3440 0.000066 0.0038 1 0.13736 0.13613 65.11 111.41 0.0043 -0.000514 snap 6898: 0.00115 57.7394 0.000063 0.0036 1 0.13736 0.13621 65.25 119.55 0.0041 -0.000473 ate 7000: 0.00112 57.8553 0.000062 0.0036 0 0.00000 -0.00112 65.25 0.00 0.0040 -0.000459 evolved 7500: 0.00102 58.3904 0.000058 0.0034 0 0.00000 -0.00102 65.25 0.00 0.0038 -0.000397 =hispanics 8000: 0.00093 58.8761 0.000054 0.0032 0 0.00000 -0.00093 65.25 0.00 0.0035 -0.000346 newer 8500: 0.00085 59.3187 0.000051 0.0030 0 0.00000 -0.00085 65.25 0.00 0.0033 -0.000303 =donaldson 9000: 0.00078 59.7256 0.000048 0.0029 0 0.00000 -0.00078 65.25 0.00 0.0031 -0.000266 download 9500: 0.00072 60.0998 0.000046 0.0027 0 0.00000 -0.00072 65.25 0.00 0.0030 -0.000235 =rupert 9532: 0.00072 60.1228 0.000046 0.0027 1 0.13736 0.13665 65.38 191.85 0.0030 -0.000240 doll 9719: 0.00070 60.2550 0.000045 0.0027 3 0.41209 0.41139 65.80 591.23 0.0029 -0.000248 hello 10000: 0.00067 60.4471 0.000043 0.0026 0 0.00000 -0.00067 65.80 0.00 0.0029 -0.000232 pounding 14668: 0.00037 62.7514 0.000030 0.0019 3 0.41209 0.41172 66.21 1116.77 0.0020 -0.000102 snaps 20000: 0.00022 64.2900 0.000022 0.0014 0 0.00000 -0.00022 66.21 0.00 0.0014 -0.000042 insolvent 20159: 0.00022 64.3256 0.000022 0.0014 4 0.54945 0.54923 66.76 2463.90 0.0014 -0.000052 rabbits 30000: 0.00011 65.8841 0.000014 0.0010 0 0.00000 -0.00011 66.76 0.00 0.0010 -0.000013 =ft -------- # Ratio of actual usage to standard usage for common words: 0.42 was 0.52 in 0.59 it 0.59 a 0.60 that 0.64 are 0.72 out 0.85 his 0.91 to 0.94 the 0.95 into 0.96 for 1.00 with 1.44 had 1.54 three 1.55 and 1.65 new 1.69 two 1.77 good 1.77 but 1.83 have 2.37 this 2.53 her 2.55 is 2.62 will 2.78 by 3.00 days 3.19 came 3.40 can 3.40 what 3.51 so 3.58 all 3.64 he 4.16 at 4.18 just 4.20 house 4.20 go 4.37 now 4.39 may 4.43 city 4.48 see 4.87 one 5.13 say 5.31 said 5.36 wanted 5.47 not 6.43 white 6.73 do 6.99 like 7.26 run 7.83 got 8.09 play 8.10 morning 8.70 best 8.84 did 9.20 she 9.20 i 9.66 open 9.71 little 9.93 stay 10.54 big 10.69 home 10.96 then 11.10 am 11.34 too 11.85 me 13.08 find 13.14 comes 13.29 you 13.54 gave 15.24 away 15.33 red 16.06 again 16.27 mother 16.70 here 16.98 brown 18.37 man 20.25 went 20.30 green 20.61 want 21.29 old 21.65 come 21.83 walked 21.94 my 22.62 ball 23.22 name 25.93 blue 26.58 look 27.55 something 31.20 animals 31.83 likes 35.72 walk 36.83 stopped 38.15 liked 40.72 saw 42.14 girl 48.76 pretty 49.72 store 50.66 box 53.10 ride 53.19 yes 61.29 garden 63.05 window 69.99 happy 88.61 surprise 96.62 wish 100.39 train 111.41 snap 119.55 ate 119.78 oh 132.24 mr 137.25 dog 143.93 ran 163.92 breakfast 189.31 boat 191.85 doll 203.30 gate 235.05 looked 244.16 bird 250.61 eat 310.78 toy 408.99 birds 505.01 laughed 591.23 hello 649.69 coat 1116.77 snaps 2463.90 rabbits -------- # 0 rare legitimate tokens, 0 rare legitimate token types: -------- # 0 rare tokens (0 token types) were ignored: -------- # 98 illegitimate tokens, 25 illegitimate token types: 30 =Alice 2 =AliceWhite 2 =BetsyLee 1 =BigGreenBird 1 =Bow-wow 5 =Carl 4 =Father 2 =Good-by 2 =Green 1 =IS 1 =J 6 =Jack 1 =JackRabbit 19 =Jerry 5 =Jip 3 =LittleGoat 2 =May 4 =Mother 1 =Snap 1 =bow-wow 1 =good-by 1 =ip 1 =lice 1 =n 1 =tra -------- General summary of sample text: 826 tokens of all kinds (excluding punctuation); trailing tokens were: 728 legitimate tokens (including woodpecker exceptions) 155 sentences 128 sentences ending in periods (82.6 percent) 4 sentences ending in question marks (2.6 percent) 23 sentences ending in exclamation marks (14.8 percent) 0 sentences ending in "@" (0.0 percent) 5.3 tokens/sentence average 5.0 tokens in median-length sentence 12 tokens in longest sentence Most overused words are: rabbits(2463), snaps(1116), coat(649), hello(591), laughed(505), birds(408), toy(310), eat(250) For grammatical words: token count = 269 token types = 20 aggregate percentage of legit tokens = 36.95% aggregate percentage of total tokens = 32.57% mean U-value = 19640.8761 right boxes ABSOLUTE area = 20.95 right boxes NET area = 1.42 total contribution = 21.9135 For common_content words: token count = 459 token types = 107 aggregate percentage of legit tokens = 63.05% aggregate percentage of total tokens = 55.57% mean U-value = 837.5071 25% rank = 64.0 25% rank uvalue = 1601.7900 50% (median) rank = 183.0 50% (median) rank uvalue = 507.2300 75% rank = 1202.0 75% rank uvalue = 95.2000 right boxes ABSOLUTE area = 137.65 right boxes NET area = -25.86 total contribution = 18.7515 For common words: token count = 728 token types = 127 aggregate percentage of legit tokens = 100.00% aggregate percentage of total tokens = 88.14% mean U-value = 7785.4553 right boxes ABSOLUTE area = 158.60 right boxes NET area = -24.44 total contribution = 40.6650 For legitimate words: token count = 728 token types = 127 aggregate percentage of legit tokens = 100.00% aggregate percentage of total tokens = 88.14% mean U-value = 7785.4553 right boxes ABSOLUTE area = 158.60 right boxes NET area = -24.44 total contribution = 40.6650 For all_content words: token count = 459 token types = 107 aggregate percentage of legit tokens = 63.05% aggregate percentage of total tokens = 55.57% mean U-value = 837.5071 right boxes ABSOLUTE area = 137.65 right boxes NET area = -25.86 total contribution = 18.7515 For rare legitimate words: 0 rare tokens ignored due to "Woody Woodpecker" exception (0.0000% of total tokens) token count = 0 token types = 0 aggregate percentage of legit tokens = 0.00% aggregate percentage of total tokens = 0.00% For illegitimate words: token count = 98 token types = 25 aggregate percentage of total tokens = 11.86% Weighted sum of frequency differences: -0.186137