January 11, 2003 - The BiggestWiki page [L1 ] was updated, but how can one get an updated count of the number of pages in this wiki?
Not easily... I've looked at it with MK database cmds: 6253 page entries, of which 2595 are empty (never filled in) -jcw
June 20, 2002 - obsolete and confusing info from this page has been removed -jcw
Performance of wiki markup rendering used to be pretty bad (see comment further down). The kiwi project includes a new renderer which is probably 10x faster (but only HTML so far, no Tk version).
It's all largely irrelevant for CGI use, now that wiki uses a static page cache. Things are more than fast enough.
Having just converted to new hyperlink tracking code in wikit, I decided to collect a bit of stats. First the output (updated Nov 15, 2002):
4605 pages 1476 pages have never been filled in 9794 archive entries 16487 hyperlinks 2782 pages have hyperlinks to others 2838 pages are being hyperlinked from others 260 leaves (pages not referencing any others wiki pages):
12 33 45 80 94 98 110 112 114 115 117 119 120 124 126 130 133 134 135 142 143 157 158 160 169 186 195 205 287 288 289 292 293 306 307 310 311 315 316 318 319 326 330 331 352 368 370 372 374 381 408 416 422 424 426 429 431 433 474 475 476 558 568 570 574 575 589 616 629 636 651 670 673 675 678 679 680 681 710 716 730 762 767 785 790 811 816 832 841 842 858 865 896 905 928 939 1087 1095 1098 1135 1142 1144 1172 1173 1174 1190 1192 1201 1210 1219 1240 1244 1265 1285 1289 1342 1350 1357 1359 1521 1528 1541 1610 1629 1634 1641 1649 1654 1747 1804 1826 1828 1833 1885 1891 2040 2059 2063 2111 2125 2127 2128 2132 2134 2137 2152 2156 2168 2194 2247 2250 2346 2377 2386 2387 2407 2416 2458 2497 2515 2535 2539 2558 2606 2661 2701 2754 2815 2861 2939 2940 2975 3001 3019 3033 3050 3064 3066 3103 3115 3122 3182 3288 3289 3294 3317 3405 3406 3435 3438 3442 3449 3472 3473 3476 3491 3536 3543 3545 3585 3664 3671 3686 3703 3762 3763 3828 3843 3872 3877 3880 3881 3898 3965 3975 3983 4018 4031 4034 4040 4041 4045 4065 4069 4076 4096 4134 4140 4148 4192 4208 4238 4277 4301 4310 4311 4314 4315 4316 4318 4320 4322 4341 4342 4347 4368 4371 4382 4392 4431 4450 4469 4480 4481 4482 4499 4500 4501 4502 4540
281 orphans (pages not referenced from any others):
23 65 151 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 328 329 384 385 386 387 388 389 390 394 404 412 415 419 432 438 439 449 452 455 456 464 465 466 467 520 521 522 536 538 545 554 572 577 578 579 580 581 582 597 598 599 601 605 641 655 658 664 665 695 700 705 725 726 742 745 746 753 754 789 798 802 806 807 810 817 818 845 854 862 867 868 892 893 899 914 925 944 965 1005 1068 1076 1131 1136 1138 1139 1160 1169 1209 1216 1221 1226 1229 1231 1238 1250 1257 1271 1282 1284 1288 1311 1319 1320 1338 1341 1349 1352 1370 1371 1376 1377 1382 1523 1524 1676 1706 1768 1774 1795 1839 1843 1858 1876 1907 1920 1929 1930 2121 2182 2184 2209 2213 2225 2388 2410 2412 2427 2438 2454 2474 2481 2505 2511 2521 2522 2526 2534 2568 2646 2654 2669 2700 2733 2742 2764 2800 2840 2847 2873 2880 2934 2938 2972 2998 3013 3017 3029 3045 3046 3049 3059 3072 3073 3088 3101 3136 3147 3160 3165 3168 3185 3196 3198 3207 3213 3214 3230 3231 3246 3247 3262 3273 3285 3297 3310 3314 3328 3329 3335 3339 3357 3391 3398 3400 3402 3434 3436 3437 3469 3470 3481 3492 3504 3587 3590 3642 3658 3693 3695 3696 3718 3719 3725 3727 3775 3799 3806 3816 3820 3995 3999 4008 4033 4092 4160 4598
The code that generated the above:
#! /usr/bin/env tclkit
mk::file open db wikit.tkd -readonly
puts " [mk::view size db.pages] pages"
puts " [llength [mk::select db.pages date 0]] pages have never been filled in"
puts " [mk::view size db.archive] archive entries"
puts " [mk::view size db.refs] hyperlinks"
mk::loop c db.refs {
lassign [mk::get $c from to] f t
if {$f < 10 || $t < 10 || [mk::get db.pages!$t date] == 0} continue
if {![info exists froms($f)]} {
set froms($f) 0
}
incr froms($f)
if {![info exists tos($t)]} {
set tos($t) 0
}
incr tos($t)
}
puts " [array size froms] pages have hyperlinks to others"
puts " [array size tos] pages are being hyperlinked from others"
set leaves {}
foreach {k v} [array get tos] {
if {![info exists froms($k)]} {
lappend leaves $k
}
}
puts " [llength $leaves] leaves (pages not referencing any others wiki pages):"
puts [join [lsort -integer $leaves] "\t "]
set orphans {}
mk::loop c db.pages 10 {
set i [mk::cursor pos c]
if {[mk::get $c date] != 0 && ![info exists tos($i)]} {
lappend orphans $i
}
}
puts " [llength $orphans] orphans (pages not referenced from any others):"
puts [join [lsort -integer $orphans] "\t "]06oct02 jcw - Wikit has been modified to enhance its performance when rendering pages with lots of embedded page references, including pages such as Recent Changes. This has a more-than-order-of-magnitude effect. This is especially noticeable in local mode, but CGI also benefits.
escargo - 19 Nov 2002: Are there any built-in functions for trimming the unfilled pages? To me it seems like many of them must have been created by accident, probably by new people who created pages with typographical errors in their names. I would think that a mark/sweep or generational approach, marking pages as dead in one phase, and then reaping them in a later phase, would be both straight forward and useful.
A function to generate a list of all [links] that point to completely blank pages might be nice as well.
Sure ... <insert sound of flexing fingers> ... here ya go!
mk::file open db wikit.tkd -readonly puts [mk::select db.pages page ""]
Remember, though, they aren't necessarily candidates for immediate deletion - the refs view has to be sure not to contain any links to those pages first.
jdc 9-nov-2007
Running the above script (archive lines remove) on a recent wikit.tkd results in the following:
20152 pages 10454 pages have never been filled in 77826 hyperlinks 9115 pages have hyperlinks to others 8822 pages are being hyperlinked from others 288 leaves (pages not referencing any others wiki pages): 331 422 423 424 426 427 431 433 514 568 570 574 575 589 629 651 655 665 744 767 785 816 832 905 1087 1095 1289 1311 1352 1357 1509 1528 1541 1610 1653 1747 1778 1779 1780 1782 1783 1785 1786 1804 1868 1891 1929 2017 2040 2111 2118 2152 2156 2194 2209 2416 2458 2481 2487 2539 2810 2939 2975 3088 3115 3294 3353 3435 3438 3472 3473 3476 3664 3881 3975 4040 4278 4310 4311 4314 4315 4316 4318 4322 4342 4371 4450 4469 4480 4499 4500 4540 5118 5193 5241 5731 5790 5903 5974 6030 6031 6050 6088 6263 6577 8331 8345 8377 8383 8416 8439 8631 8651 8734 8741 8842 8858 8874 9044 9059 9189 9243 9296 9369 9385 9445 9543 9565 9567 9579 9659 9682 9730 9766 9804 9807 9850 9989 9990 10128 10256 10324 10382 10389 10413 10496 10546 10555 10676 10686 10729 10738 10748 10753 10754 10756 10987 11012 11286 11302 11307 11497 11609 11630 11646 11662 11689 11723 11807 11854 11862 11863 11997 12000 12083 12126 12142 12181 12205 12544 12582 12697 12698 12712 12834 12843 12859 12886 12964 12979 12981 13011 13012 13063 13108 13127 13291 13333 13339 13580 13732 14371 14380 14451 14453 14501 14503 14512 14515 14699 14724 14725 14757 14763 14826 14830 14838 14862 14896 14925 14953 14993 15015 15020 15123 15145 15174 15246 15275 15334 15343 15504 15562 15572 15575 15628 15698 15759 15845 15984 15989 15998 15999 16035 16073 16093 16131 16202 16297 16329 16620 16621 16647 16688 16707 16889 16947 17128 17159 17237 17249 17323 17324 17361 17480 17509 17526 17602 17624 17707 17753 17974 17999 18009 18058 18143 18193 18436 19606 19656 19658 19664 19673 19815 19874 19880 19888 20151 869 orphans (pages not referenced from any others): 32 33 41 59 80 152 230 233 234 235 236 240 241 242 244 245 247 249 250 252 254 255 256 259 261 263 264 265 266 268 269 385 386 387 388 389 390 394 404 412 419 438 449 452 455 456 464 465 466 467 492 494 520 521 522 538 542 545 554 577 578 579 580 581 582 597 598 599 612 641 658 664 671 674 742 745 802 806 807 810 841 867 892 899 921 947 965 1068 1076 1122 1124 1138 1139 1164 1169 1231 1319 1320 1338 1523 1524 1585 1697 1731 1768 1839 1911 2093 2144 2167 2219 2225 2427 2498 2505 2511 2577 2669 2742 2754 2764 2774 2903 2921 3036 3046 3121 3136 3207 3209 3318 3329 3335 3342 3434 3436 3437 3471 3491 3525 3560 3658 3695 3718 3763 3772 3818 3965 3969 3979 3983 4028 4160 4178 4185 4230 4251 4259 4329 4482 4490 4527 4605 4606 4747 4784 4823 4847 4881 4939 4962 4979 5040 5043 5054 5061 5062 5076 5078 5079 5082 5123 5128 5160 5186 5230 5268 5273 5295 5406 5419 5460 5461 5597 5628 5633 5635 5636 5638 5639 5641 5648 5703 5725 5873 5929 5930 5931 5952 5991 6010 6089 6099 6102 6131 6147 6240 6260 6297 6305 6312 6320 6370 6466 6518 6570 6589 6598 6622 6651 6678 6779 6831 6839 6847 6871 6894 6938 6966 6997 7007 7092 7093 7106 7146 7215 7302 7307 7382 7418 7440 7496 7524 7525 7541 7641 7668 7715 7751 7781 7886 7937 8058 8168 8181 8207 8317 8355 8356 8372 8434 8504 8506 8508 8544 8650 8690 8701 8736 8773 8785 8848 8851 8859 8870 8877 8879 8897 8916 8917 8928 8932 8992 9005 9017 9057 9083 9110 9142 9192 9241 9255 9266 9285 9295 9300 9320 9329 9350 9473 9492 9496 9564 9580 9629 9640 9651 9667 9694 9705 9740 9845 9858 10032 10034 10039 10130 10280 10297 10310 10317 10337 10341 10412 10499 10501 10503 10569 10603 10606 10645 10653 10662 10665 10666 10667 10671 10672 10673 10680 10695 10739 10773 10873 10895 10946 11004 11076 11089 11106 11108 11181 11243 11278 11298 11334 11364 11390 11409 11457 11463 11464 11467 11468 11469 11480 11481 11485 11491 11546 11565 11614 11629 11638 11756 11757 11763 11764 11802 11809 11836 11837 11844 11874 11885 11925 11926 11932 11937 11966 11977 12069 12071 12106 12125 12216 12223 12233 12258 12259 12263 12335 12394 12396 12417 12474 12484 12511 12536 12540 12541 12552 12553 12556 12585 12677 12678 12696 12713 12799 12806 12819 12826 12835 12866 12868 12870 12877 12878 12880 12881 12884 12888 12892 12896 12904 12906 12907 12915 12926 12927 12961 12962 12968 12972 13001 13003 13006 13020 13035 13044 13060 13102 13159 13167 13227 13322 13334 13345 13366 13367 13383 13406 13420 13468 13470 13492 13511 13564 13566 13579 13612 13625 13627 13628 13633 13649 13650 13663 13664 13665 13673 13694 13703 13714 13717 13718 13730 13752 13759 13762 13766 13768 13816 13917 13926 13950 13978 14012 14069 14083 14085 14099 14102 14134 14144 14254 14302 14312 14319 14374 14397 14423 14486 14494 14499 14511 14528 14533 14536 14541 14558 14568 14589 14627 14628 14651 14679 14767 14769 14782 14790 14805 14827 14892 14904 14984 14992 15007 15019 15058 15059 15119 15133 15136 15138 15168 15175 15179 15187 15235 15241 15255 15257 15291 15311 15329 15340 15342 15357 15359 15365 15366 15367 15368 15369 15371 15373 15374 15377 15378 15380 15381 15382 15383 15384 15391 15420 15429 15550 15616 15617 15618 15624 15634 15637 15645 15684 15692 15706 15716 15721 15749 15856 15890 15924 15952 15964 15972 15976 15991 16007 16012 16029 16040 16065 16075 16097 16103 16110 16112 16113 16147 16215 16248 16250 16267 16269 16271 16273 16279 16284 16286 16291 16292 16318 16326 16657 16670 16678 16679 16723 16739 16992 17003 17025 17046 17098 17113 17118 17127 17170 17182 17213 17214 17216 17217 17218 17219 17220 17221 17222 17223 17224 17226 17227 17228 17229 17230 17231 17232 17233 17234 17235 17236 17238 17240 17241 17242 17243 17244 17245 17246 17247 17248 17250 17266 17273 17280 17287 17290 17332 17333 17334 17345 17364 17406 17435 17466 17483 17490 17494 17500 17501 17502 17503 17504 17505 17506 17515 17518 17520 17528 17529 17531 17532 17533 17534 17535 17536 17537 17538 17539 17544 17545 17546 17547 17548 17549 17550 17552 17553 17555 17562 17563 17564 17565 17566 17567 17568 17570 17571 17572 17573 17574 17575 17576 17577 17578 17579 17580 17581 17582 17583 17584 17586 17587 17588 17589 17591 17605 17607 17647 17706 17745 17787 17789 17791 17792 17793 17794 17795 17796 17797 17798 17806 17809 17810 17811 17812 17815 17817 17818 17826 17830 17831 17832 17833 17834 17835 17836 17837 17838 17839 17840 17857 17899 17902 17903 17904 17905 17906 17907 17908 17909 17910 17911 17912 17913 17914 17915 17916 17918 17919 17920 17921 17928 17929 17930 17931 17932 17933 17935 17936 17965 17969 17989 18005 18006 18010 18011 18014 18015 18016 18017 18019 18070 18080 18107 18145 18155 18158 18203 18204 18438 18444 18473 18474 18475 18476 18477 18478 18758 18761 19341 19595 19622 19636 19665 19667 19686 19695 19697 19704 19709 19889 19956 20070 20098