Sunday, 11 August 2013

Moving to a faster DB server

Down-time was experienced today due to moving to a faster database server. I'm hoping this will help the performance of NoobMeter, as the number of people using it and thus the database size seem to continuously increase.

Please let me know in the comments if you think performance is an issue or not, and on which functions in particular.

If I don't hear from you, I will focus more on new features rather than performance optimization.

110 comments:

  1. Never had any encountered any major problems in performance here.

    As for features, I would love to see player comparisons in last 1000 battles or even better, 2000 battles. Something I've been waiting for.

    ReplyDelete
    Replies
    1. I like this idea, I will try to find time to implement it.

      Delete
    2. Hi Not sure where to post so sorry if in incorrect location. There seems to be a issue with the stats. I know that I played more than 4 games yesterday but it is being reported as 4. I suspect that it has something to do with swapping servers. so the totals are only for one server played on.

      Delete
    3. aardv, I think it is regarding swapping servers, but nothing I can do about it.

      WG may return slightly outdated data as they have some lag in replicating results between servers. NM cannot get the latest data - but the battles will appear eventually.

      Sorry about this.

      Delete
    4. Player comparison for recent battles has been added. Thanks again for suggesting this.

      Delete
  2. Hi, I've been wondering about something for months. Why doesn't my overall WN6 and WN7 ratings seem to change to match my most recent values. My overall WN6 is 585 (Bad) and my WN7 is 566 (BAD), but in the last 839 battles they were 707 and 809. In the last 7 days they were 753 and 892, and in the last 1 day they were 1408 and 1528. I don't think my actual WN6 or WN7 has been as low as the'overall' figures in many months, yet they stay so low. Why is that??

    ReplyDelete
    Replies
    1. I suspect it could be because WN6/WN7 overall ratings get hit by the low-avg-tier penalty, but I don't take it into account for interval ratings because otherwise very strange results occur.

      In any case, I haven't developed WN6/WN7 so cannot give an authorative answer, try Praetor77 from NA forums.

      Is PR behaving better?

      Delete
  3. I had a question, and didn't see it answered anywhere else on your blog: are Noobmeter subscribers only allowed to upload battle replays that end in victory?

    Thanks, and keep up the great work!

    ReplyDelete
    Replies
    1. No, defeats are fine too.

      Here are a number of defeats uploaded:

      http://www.noobmeter.com/replaySearch/all/all/all/all/all/false/onlyLost/nominalXp

      Delete
    2. Thanks for the quick reply! I'll try again to upload a few replays I've been having problems with. Love Noobmeter!

      Delete
    3. Lou, what it could have trouble with are replays where the "hero" (so to speak) leaves in the middle of the match. Then all the information is not recorded.

      But if you have any specific replays NM is not accepting, send them to me somehow and I can investigate specifics.

      Delete
  4. 'SEA Server' from 'KR Server' has moved to the account.
    But 'NoobMeter' is not updated.
    It was version 8.5.

    ReplyDelete
  5. My clan data was messed with china server clan


    http://www.noobmeter.com/clan/na/FLA;jsessionid=68BDCCAC509A54A7CAFC29A97F972293

    I saw a chinese player which is one of [ADOLH] in our First Legion Asiania [FLA] list

    the page title should be Players for clan FLA (First Legion Asiania) on server NA

    not Players for clan FLA (【虎鯨突襲】) on server NA

    ReplyDelete
    Replies
    1. Thanks for the bug report. I will fix this ASAP.

      Delete
    2. The clan IDs are overlapping so I shifted the Chinese clans but not by enough, so they ended up overlapping your clan.

      But I have fixed it now, they are no longer in your clan's list.

      Sorry about the inconvenience.

      Delete
  6. I found a bug regarding sorting. When I enter my clan's page (ISOUL in EU server) and sort by percentage descending than e.g. 59.0% of wins is higher than 59.7% and other 59.x% numbers are sort in a random way

    Not that I find it a major bug, but it's something that can be fixed easily, i think.

    http://s22.postimg.org/djgdstrf5/ISOUL.jpg

    ReplyDelete
  7. How do you count company battles? Why does this feature works for any accounts and for another not?

    Thanks in avdance,
    bfcmyxa

    ReplyDelete
  8. I tried to play a replay downloaded from here, but it would not as the version was different. You might want to add the client version as a field in the description of the replay

    ReplyDelete
  9. I've not noticed too much of a performance issue. I will say this, that my clan stats page still shows many members who are no longer in Total Oblivion. If other clans are experiencing the same thing, that could account for the huge database. If you could pull from WoT only the current members as of date, this would cut out at least 8 members from my clan. 8 times however many clans you see pulling your db...could be a ton of clearance for your db performance issue.
    Thanks much for your site. Keep up the great work.

    ReplyDelete
    Replies
    1. Also, maybe delete those that have no 60+ day activity? Although, I've noticed half my members don't display a 30 nor 60 day, yet do display overall, which is a bit confusing considering they've been active for awhile. hrm

      Delete
    2. FYI, I've noticed a pattern as to why former members still show on our clan list. It seems members who leave the clan are cleared off our clan list. However, those we remove from our clan still remain. Must be separate dbs somewhere.

      Delete
    3. Actually, if you check a person in NM and his clan is outdated then he will get his clan updated (although it may be cached for a few hours still).

      However, if nobody checks a person, then NM doesn't really find out on his own that he has left a clan, so he may remain on clan page.

      This is how it should be working now, you can check by clicking on a few of the guys no longer in your clan.

      But I agree a better solution is needed.

      Delete
    4. Thanks NM. I've clicked on the ones we've let go again to see if they will update.

      Delete
    5. Tell me if that fails, then I can allocate time to do something more fancy.

      I can contact go through WG's clan list and compare with the one in my DB, but it is another piece of code to write, and another set of requests against WG servers, so I'm reluctant to do it.

      Delete
    6. Understood. It is slowly updating. Takes about 3 days for them to finally be removed from the list. some still show. but it's all good. love the site. ty

      Delete
  10. I've been watching my noobmeter page for a while now, but it doesn't seem to want to update my last 5 battles that I played like a week ago or so. Is 5 battles too small for the database to bother updating?

    ReplyDelete
    Replies
    1. Generally yes. But if you link your account with NM then it will update every night, even 5 battles.

      Delete
  11. They seem to have fixed the missing premium tanks. At least on the profile pages.

    Just leaving this here to guarantee that you notice. :P

    ReplyDelete
    Replies
    1. Thanks for letting me know.

      Yes, they have added M60-ultra-rare-t10-med and M6-goose-t8-heavy to the feeds. But they are still missing M4-slow-premium-Sherman.

      Delete
    2. The M4A2E4 Sherman was also added, I think the KV-220 thing as well. Check my profile and you can see it.

      http://worldoftanks.eu/community/accounts/500149056-Dead3yez/

      M60 and M6whatever where fixed earlier (2nd/3rd of sept), if I understand correctly then the API will update later today (but I dont really know about all that stuff).

      http://forum.worldoftanks.eu/index.php?/topic/290080-minsk-wtf-are-you-doing-a-wg-support-story/page__st__40__pid__5790554#entry5790554

      It all got fixed once the right person had got the attention of this thread. After several months it all took less than a day to get some of those missing stats back.

      Delete
    3. Actually, you are right, web page has the M4A2E2 Sherman for persons for whom they are not in NM yet. I hope API will have it soon then.

      Thanks for pointing me to that link, quite interesting. Impressive that KawaGreen spent so much time arguing them, much respect to both him and gustav_fuchs for this.

      Delete
    4. Seems to be fixed. Recent stats a bit borked though. :P

      Delete
    5. Indeed. For my recent battles i get a nice black diamond with a white question mark (�)

      Delete
    6. This should resolve itself when the old corrupt snapshots will be removed (over time).

      The problem is that if A-32 wasn't available in one snapshot and then suddenly all the battles (407 in your case) appear, then NM thinks you played 407 battles without doing any damage. That is not good for your ratings :).

      Delete
  12. Hello,

    Is there any way to allow users to set preferences to a local time reference? I don't see one...I don't know if it's a feature anybody really cares about, but I find it to be very helpful to accurately interpret my stat tracking...

    ReplyDelete
    Replies
    1. I implemented something that should help (automatic timezone detection basically).

      Let me know if there are any issues with that.

      Delete
  13. Hi there,

    For the Tank Company statistics, where are you pulling that from? Just curious because when they first started it said I had 1100 TC battles (felt a bit high) and now it just doesn't show up at all on my profile (un: AMartin223, NA server). Thoughts?

    ReplyDelete
    Replies
    1. There's a new WG beta 2.0 API which will become official upon 8.8.

      However, it often returns errors to NM so if it has returned an error when your stats were pulled then your TC battles would be missing again. Hopefully next time it will not return an error and they will reappear.

      Let me know if that doesn't happen, I can look into it.

      Delete
  14. Hey there. I have noticed the last week that my Noobmeter account is updating several times a day now. Is this a new change? It's making it impossible for me to see my average performance over 24 hours which is a bit of a shame. Working as intended?

    Love the website :).

    ReplyDelete
    Replies
    1. It is a new change as people were saying it doesn't update often enough, however it should still be possible to see your average performance over 24 hours.

      Send a link to your account and I will see what is happening.

      Delete
  15. Ahhhh, I see. My account is here: http://www.noobmeter.com/player/eu/SirHedgeKnight/508259672/

    Thanks :)

    ReplyDelete
    Replies
    1. Ser,

      I suppose if you select the right periods you can still see results over 24 hours:

      E.g.
      http://www.noobmeter.com/recent/eu/SirHedgeKnight/508259672/508259672_00003650_20130907_173634/508259672_00003670_20130908_163758

      Delete
  16. Don't know how to contact you directly so let you know here - the data at http://www.noobmeter.com/tankStats/eu seems incomplete - there are OP tank ratings based on only 0k battles.

    ReplyDelete
    Replies
    1. Thanks for the report, I am investigating this.

      Delete
  17. Hello,
    i have the following problem with noobmeter:
    My stats have been tracked until 14.03.2013, 2942 battles. A short time after that, i had an account reset (at 3025 battles). Back then i wasn't registered here.
    Since that reset i have played more than 1600 battles, and i am registered now (for a week or so), but still my stats don't update. There are still the 2942 battles shown, i had in march. Is this because of the account reset, or am i doing something wrong?
    I hope, my question ist understandable, i'm not a native speaker...

    ReplyDelete
    Replies
    1. Tell me your server and username and I will reset your NM account.

      Delete
  18. Wow, thanks for the fast reply.
    My username is Caedmon, i play on EU1.

    Thanks and keep up the good work!

    ReplyDelete
    Replies
    1. Removed your NM account. Old one may be cashed so check again in a few hours.

      Delete
  19. The Tank Stats seem to be COMPLETELY broken, with 0K recent battles on most tanks (even very commonly played tanks), 1-5K total battles on most tanks, many tanks not showing up at all and tanks like the Vickers Mark II and Matilda Black Prince reporting impossible OP ratings of 1,000+

    This is a pity, as I was hoping to take a snapshot of the performance of the KV-1S, Super Pershing and the various German tanks that are getting nerfed, then compare those stats to the ones showing up post-8.8

    ReplyDelete
    Replies
    1. They are. It is a pity.

      I just changed some parameters and relaunched the calculation. It will either work (good) or it won't (bad) - in the latter case I can restore the tank stats to their previous state before they were broken.

      Delete
    2. I don't suppose you could send me a screenie or something of what they looked like before they were broken, could you?

      Delete
    3. Unfortunately I have no screenshot, I just have a database table where the generated stats are logged, but no UI to read it yet.

      Delete
    4. So I'm just going to have to keep needling you until you find a way to display the information, then? :P

      Delete
    5. I think the stats have been fixed now. Let me know what you think.

      Delete
    6. Yay! Thankyou so much! ^^

      Hehe, I remember the E-100 had pulled just ahead of the Maus (but only by the very narrowest of margins) just before the stats broke, so it seems even the little things have been preserved. Nice job. :3

      Have already saved a snapshot of the current stats.

      Delete
    7. Although call me pedantic, but I preferred it when the OP Ratings were to two decimal places. XD

      Delete
    8. Haha. I wish, but I'm running out of space on screen.

      Delete
  20. Hi, your site is great & it's helped me track my progress no end, thank you. But I have a serious concern about the fact I play as scout. Your algorithm seems to screw me three times. Firstly because damage spotted isn't included (but I appreciate there's nothing you can do about that coz that data isn't supplied by WG). But also - secondly because scout tanks don't generally cause much damage themselves but die a lot, and thirdly because scout tanks are inherently lower tier.

    On my account (MaxMk6 on EU server), I've played 13,218 games. 1,151 in my tier 3 T15 (that has a tier 2 gun & +2 mm spread), 515 in my tier 4 Luch, 115 in my tier 4 Pz38NA, 93 in my tier 4 M5 Stuart, and 803 in my tier 5 Leopard. The Luch, Pz38NA, M5 Stuart, and Leopard all get +4 mm spread. None of those tanks are seal clubbers and because of mm, none of those tanks can do significant damage, they are purely scouts.

    I'm worried that playing as scout is making me look like I'm rubbish at doing damage and am a seal clubber when that isn't true. Although I'm not a great scout or player, I'm OK at scouting, as shown by the fact that on my WOT's profile it says I'm placed as 67,978 for battles participated, but 10,347 for enemies spotted. It's also shown by the fact that I have 146 patrol medals & 168 scout medals.

    Because of this I've been accused in game (I suppose by people using XVM) of having bad stats. This is worrying me to the point where I'm seriously considering giving up scouting altogether. I've decided not to buy a Chaffee and have already sold my M5. I'm also on the verge of selling my T15, Luch & Leopard. I'm also considering starting a new account because of the damage scouting has already done to my stats and apparent performance.

    It has definitely affected my efficiency rating, which is showing as average despite my WN7 & noobmeter performance showing as good. My question is, am I worrying to much about how much this is affecting my performance rating? And, is it worth considering amending your algorithm to avoid classifying tier 3, 4, & 5 scout tanks as sealclubbers? Thank you

    ReplyDelete
    Replies
    1. I agree with the points you make. The damage requirements for scouts aren't very high, but I agree that judging scouts by damage isn't very good.

      I have some ideas how to solve this (e.g., consider winrate more in case of scout tank players, or not having them be affected by low-tier penalty as you suggest), but I have no idea when I will have time to actually implement them, sorry.

      In the meantime one can hope that people viewing your profile realize you are mostly a scout player and thus don't judge your ratings too harshly.

      Delete
    2. Even leaving aside the fact that you never, EVER sell premium tanks, you'd be foolish to give up scouting. Not only are decent scout players are an extreme rarity in World of Tanks that we desperately need more of, but not playing the tanks you enjoy playing because it's causing some numbers on a screen to shift the wrong way smacks of missing the forest for the trees.

      If anyone accuses you of having bad stats because they don't know how to use XVM, just go ahead and let them be retarded. What in matters in WoT, as in any game, is whether you yourself are enjoying yourself, and you obviously enjoy playing scout tanks. So keep playing them! I've been accused of statpadding because I have a lot of games in my Matilda, but I just ignore them and play my goddamn Matilda, because fuck it, the Matilda is hella fun and I don't care what my stats look like. There's a reason the Char B1 is one of my most played tanks!

      There will always be naysayers. Giving up your favorite tanks isn't going to change that. It's not worth giving up your account on the whim of people who will call you a retard regardless of what you do, because they are retarded themselves.

      Just play the tanks you enjoy and have fun doing it. Although, if you want to get your average per-damage up, I'd recommend the British TDs from tier 5-8 (yes, including the AT 2 with the 6 pdr) and probably tier 9, though I don't actually have a Tortoise yet myself.

      Delete
    3. Very well written indeed.

      Also:
      "It's not worth *doing anything* on the whim of people who will call you a retard regardless of what you do, because they are retarded themselves."

      This point applies not just to tanks.

      Delete
  21. Dear NoobMeter
    I have a problem. When I check my user name KoLove in VN sever (http://www.noobmeter.com/player/vn/KoLove/2500013519) NoobMeter responded "The following errors occured: WG server returned error". Please help me to fix it.
    Thank you.

    ReplyDelete
    Replies
    1. http://portal-wot.go.vn/uc/accounts/2500013519/api/1.3/?source_token=Intellect_Soft-WoT_Mobile-site

      This is your API page on WG side. It returns an error. Unfortunately I don't know why.

      You can maybe write to support and ask why, because API pages of other Vietnamese players work.

      Sorry I don't have a better suggestion.

      Delete
    2. Dear NoobMeter
      I have same problem with this guy. Im from VN sever too, and my user name is LittleDrgon. Can you tell me what is the link my API page? then for this problem, where should I ask for support? VN publisher or WG
      Thanks

      Delete
  22. This comment has been removed by the author.

    ReplyDelete
  23. Hey,

    So it appears that Noobmeter sometimes report the wrong tank for most recent battles.

    If you look at the link below, you can see that it shows 8 battles, all in the Hellcat. In that time period though, there should have been 4 battles in the T-80 (tier IV light tank).

    Not sure where it's getting 8 battles and that win ratio from either, 'cause I'm counting 6 battles in game. 5 wins, 1 loss.



    I've had this happen before, where it has showed a lot of battles in the M4A2E4 when there shouldn't have been any. Any thoughts?


    http://www.noobmeter.com/recent/eu/Ludd/500103569/500103569_00004366_20130912_000802/500103569_00004374_20130912_032006

    ReplyDelete
    Replies
    1. Nevermind, seems there was a delay. Due to server switching I guess.

      Delete
    2. Ludd, BTW the M4A2E4 battles disappearing & reappearing is due to this issue:

      http://blog.noobmeter.com/2013/06/inconsistent-data-in-wg-feed-update.html

      Delete
  24. This comment has been removed by the author.

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
  25. Dear NOOBMETER manage

    I was Europian user but now I'm in South Korea server

    Unfortunately, NOOBMETER RATING is not working properly now

    For example

    http://www.wotcs.com/player.php?wid=3000017068

    As you can see,

    http://www.noobmeter.com/player/kr/% EB% B0% 94% EC% 9D% B4% EC% 98% A4% EC% 8A% A4% ED% 83% 80V/3000017068

    (Paste the link tells me the impossible,

    Korea server nickname "바이오스타V" is)



    Inormal operation does not active any more

    I hope you fix this soon.

    Thank you read this.

    ReplyDelete
  26. Hey mate, Couldn't find any bugs section so here it goes:
    very nice job on the plane-meter, very informative. However I found a rather strange stat...
    http://planes.noobmeter.com/recent/eu/Gentle1/500542992/yesterday
    His stats here show, he had a 300% winrate in the p40-warhawk yesterday. Now how does one go above 100%? Does this actually get accumalated when calculating the winrate, or does that then take 100% as the highest possible value? Does it work both ways? etc etc etc. ;)

    A link in case it will change: http://s1273.photobucket.com/user/Nrg_Extince/media/gentle1_300_zps85e30b25.png.html

    ReplyDelete
    Replies
    1. I suspect it is a rounding error because the plane winrates are received from WG rounded to the nearest digit.

      Delete
  27. I'm Korean server wot user.
    I have a problem same as '바이오스타' who commented on upside.
    fix me piz.

    this is my noobmeter link:http://www.noobmeter.com/player/kr/아야세/3000325061/

    and my game nickname is "아야세"

    ReplyDelete
  28. SEA Server' from 'KR Server' has moved to the account.
    But 'NoobMeter' is not updated.
    It was version 8.5.

    User ID 소방관

    ReplyDelete
  29. This comment has been removed by the author.

    ReplyDelete
  30. Dear NOOBMETER manager.

    I was SEA user but now I'm in KR server.
    Unfortunately, NOOBMETER Rating and tank datas are not updated properly now. (no updates and it's old data.)

    My KR server nickname is 작사 .

    I believe you can fix this soon.

    Thank you.

    ReplyDelete
  31. Hi, I noticed recently that there are OP (Bad) and OP (Good) columns but I couldn't find a description of what the values in these columns mean.

    If you have a minute, could you provide so info on them?

    ReplyDelete
    Replies
    1. < 46% wr - counts as bad
      > 52% wr - counts as good

      Delete
  32. Dear NOOBMETER manager.

    I was SEA user but now I'm in KR server.
    Unfortunately, NOOBMETER Rating and tank datas are not updated properly now. (no updates and it's old data.)

    My KR server nickname is Orange_Lemon .

    I believe you can fix this soon.

    Thank you.

    ReplyDelete
  33. In my lats results are incorrect avg tier, on:
    http://www.noobmeter.com/player/eu/Sedziwoj/500095364
    shows 3,50, but when u ennter u see i play 3x tier 6, 3x tier 8 (avg 7)
    http://www.noobmeter.com/recent/eu/Sedziwoj/500095364/500095364_00012463_20130917_052927/500095364_00012469_20130917_233348

    ReplyDelete
  34. This comment has been removed by the author.

    ReplyDelete
  35. hello im korean wot player 하나카나 and im having same problem as other koreans are having with the noobmeter after transfering the acount from america. would you be kind to fix this for me? thank you:)

    ReplyDelete
  36. This comment has been removed by the author.

    ReplyDelete
  37. Not sure where I should put this or if I'm not finding the section -> So in my rating it has me as below average (bad I know) but is there a way i can view the top performance rating and the worst and the brackets that make up bad, horrible, good, etc??

    ReplyDelete
  38. Dear NOOBMETER manager.

    I was SEA user but now I'm in KR server.
    Unfortunately, NOOBMETER Rating and tank datas are not updated properly now. (no updates and it's old data.)

    My KR server nickname is 추요찡

    I believe you can fix this soon.

    Thank you.

    ReplyDelete
  39. This comment has been removed by the author.

    ReplyDelete
  40. my noobmeter doesn't work at all
    I have been transfered from sea server to kor server
    my nickname is timtam
    can you fix this if you don't mind?

    ReplyDelete
  41. I have an after reset please update statistics clan history etc nickname neros2402

    ReplyDelete
  42. This comment has been removed by the author.

    ReplyDelete
  43. Dear NOOBMETER manager.

    I was NA user but now I'm in KR server.
    Unfortunately, NOOBMETER Rating and tank datas are not updated properly now. (no updates and it's old data.)

    My KR server nickname is 곰플.

    I believe you can fix this soon.

    Thank you.

    NA Server : http://www.noobmeter.com/player/na/Gomple/1004959144/

    KR Server : http://www.noobmeter.com/player/kr/%EA%B3%B0%ED%94%8C/3000355775/

    ReplyDelete
  44. curious what was the last DB you used and which DB you moved to

    ReplyDelete
    Replies
    1. MongoDB ; moved to same thing but with AWS PIOPS EBS disks (RAID10 array).

      Delete
    2. thx for the reply. good to know. i work in data warehousing and MongoDB is gaining momentum.

      Delete
  45. Dear Noobmeter,
    Patch 9.3 came out yesterday and I played many games in the Walker Bulldog M41. I finished out the day with a good Win8(3600) , according to my in game messenger. Woke up this morning to see that Noobmeter had me down for 17 battles with zero win 8 and an efficiency of 348, what gives? I know that this is not accurate. Is there something going on with the tank or the new patch? Played one game this morning for 9k Win8, 2.5k efficiency. This is disheartening as I like to play this tank but if my stats are taking that kind of a beating, I will refrain from playing it.
    Thanks.

    ReplyDelete
  46. This comment has been removed by the author.

    ReplyDelete
  47. Focus on the optimization performance like how we did it on you own check it here Kamagra http://www.kamagra-sure.com/

    ReplyDelete