Forum Search

We’re reaching end of July and Popmundo forum threads are at around 1,670,xxx. That is, 1.67 millions and counting up.

Popodeus forum search only contains a small fraction of all the threads created daily inside Popmundo. The main reason for this is our conservative indexing speed and one access node (server). Current status of indexed threads from 1,000,000 up to 1,670,xxx looks like this:

As you can see, there’s much more red than white or green. And we have to fix that! So we’re going to take a different approach to this.

I will soon start a coordinated project hopefully with at least 10 volunteers so we can get most of the missing threads into the forum index. If you’re interested, drop me a mail at alanas.anikonis@popodeus.com and I’ll send more details when things are ready. Please state in your mail who your Popmundo character is and what forums you are subscribed to, and if you would not mind subscribing to forums that you don’t usually read. After indexing is done, you can of course unsubscribe. The indexing will most likely be done via your web browser and a Greasemonkey script, so make sure you have Firefox 3.5 or later installed. We can probably get started on this sometime in August, so it’s no rush, I have a few security issues to take care of first.

Don’t hesitate. There’s not much work involved on your part. The indexing is mostly automated and you only have to leave on your computer and either watch it happen or go watch some TV :D

More forum statistics coming soon!

I wanted a quick visual of how much of the forum I have indexed… The current thread id maximum is at around 1.42 millions, and I only have 33k threads in my database. Lots of empty space in there.. ugh. I can do better, I promise!

This is my first visualization, blue dot is an indexed thread. Top left corner is id zero, and it grows to the right, 1000 pixels/threads to the right. So the 20th line represents threadids 20,000-20,999. First pixel on line 1400 is threadid 1,400,000-1,400,999 etc. Click on image for full size view.

Later I might add color to show which thread is in Turkish, English, Spanish etc. unless I make a traditional pie chart or something.

As asked in the previous blog entry, I’m posting some of the most commonly used words (terms, as Lucene calls them) used in the Popmundo forums.

Extracting the top terms is quite simple, but getting relevant data, well… this is how it looks like:

141,846  i
108,782 de
87,742   you
84,692    me
80,864    que
71,235    la
59,152    en

… ok, so number is how many times it occurs (in 646k entries). Hardly any interesting details in the top most common terms.

I shouldn’t even index those words in the first place, they’re called “stop words”, and removing them from the search database would make it much smaller. (Actually I started filtering out stop words not too long ago, so searching for them should not be possible for posts indexed in the last month or so, unless I made a configuration mistake)

Scrolling down in the list these words pop up:

vip, solo, band, rock, idea, skills, show, thread, tour, basic, money, ceo, zombie, club… (I didn’t pick any words that I didn’t understand, in Spanish, Turkish etc.)
Well, hardly anything surprising there. Zombie is mentioned 5,750 times.

Full list of 400 most common terms: popmundo_forum_top_terms_2009.txt

Okay. I’ve long decided against posting this, since it can be seen as “personally identifiable data” in a way, which it really isn’t. It’s names, numbers and pure facts.

Top active characters in Popmundo Forums

From most active to less active:

Character name | Character ID

#1 Cécile Dicker 506023
#2 Benjamin Murray 73784
#3 Yvette Nixon 1184587
#4 Chloe Barfield 1593751
#5 Tatum Feldman 1059174

#6 Nadine Actis 1143579
#7 Emelie Flamlind 727414
#8 Elisabetta Armani 149068
#9 Hillary Tempest 508983
#10 Eva Lundhskär 2909

#11 Gary St Pier 1257230
#12 Eileen Marshall 253366
#13 Hailey Carter 1179986
#14 Darrell Gill 28077
#15 Ava Chance 1333393

#16 Priscila Sampaio 71021
#17 Woodrow Sterling 64762
#18 Sara Hansen 66768
#19 Beth Christou 144972
#20 Marat Ivanov 786980

#21 Azzurra Campanella 496941
#22 Garrett Mason 240502
#23 Tess Granahan 617919
#24 Eric Cloarec 106255
#25 Tom Capelhorn 223855

There you have it. All data was gathered from the Popmundo Forum Search database.

I’m not displaying post counts, but #1 has 2.8x more posts than #25. #1 and #2 differ only by about 140 posts, #2 and #3 differ by 600 posts…

This is all from ~632,000 indexed posts.