Warrior General Games

Test Server Update 8/15/10

The new testers are now being put to work. So, with that comes some more testing, but we are currently working on some very large projects which are requiring a lot of time to test, so we don't currently have a compiled list of games and data for you just yet, but in the interest of full disclosure, Catch gave me permission to update everyone. So here's what's currently going on.

__________________________________________________________________________
The Sim Test teams are working on running/analyzing test games for the control for the SA testing and balancing. That's right folks, SA balancing is still coming. While its still in the beginning stages, the long awaited balancing is starting. We'll keep you posted on that as it develops.
__________________________________________________________________________


Meanwhile, across town:


__________________________________________________________________________
The Website Test teams are working on trying to determine where the phantom AI inputs are. Bort has coded in a new feature allowing us an AI trace on the test server so the testers are working on figuring out where these ghost plays are coming from and what is causing them. Again, when we get more information on it, it'll be posted, but this is also relatively early in the process.
__________________________________________________________________________
So, now you're all up to date with what has been going on on the test server.

I'll see what I can do to answer your questions in this thread. I'm sure some of the other testers/documenters will too when they pop in.

Please tell me you're doing some sort of regression analysis for the impact of SAs, not just watching 4-5 games for each?

Originally posted by Octowned
Please tell me you're doing some sort of regression analysis for the impact of SAs, not just watching 4-5 games for each?

right now, we're working on the control games. We haven't even begun testing the varying SAs yet. 100 games worth of test data to analyze for the control. I'll bring up the idea to catch. I'm not 100% sure as to what he has planned. I'd guess he plans on doing something to the effect of testing the SAs at varying levels, but we haven't really been told either way. I'll see what I can do to get him in here to answer the question for you, or get the answer from him and post it back up here.

Edited to fix an error

Originally posted by Octowned
Please tell me you're doing some sort of regression analysis for the impact of SAs, not just watching 4-5 games for each?

asked catch in a PM for you, his response:

"tell him we are running a 100 game base with players having no SA's for comparison purposes and that the comparison testing will also be large sample sizes."

Sounds reasonable. Also make sure you're testing each SA individually, then for stacking, then for interactions (offense vs defense), so the results can actually be attributed to something meaningful when compared to the control.

Originally posted by Octowned
Sounds reasonable. Also make sure you're testing each SA individually, then for stacking, then for interactions (offense vs defense), so the results can actually be attributed to something meaningful when compared to the control.

yeah fo sho

Originally posted by Octowned
Sounds reasonable. Also make sure you're testing each SA individually, then for stacking, then for interactions (offense vs defense), so the results can actually be attributed to something meaningful when compared to the control.

The plan we have come up with is to test each SA at 5, 10 and 15, and then to put that SA back down to 0 and move along to the next SA at 5, 10 and 15 points.

There has also been talk about doing further testing once the initial SA testing is done, particularly with opposing SAs such as monster hit and cover up. However, I'm sure you'll realize that this will take some time, and that testing for stacking and even interactions wouldn't possibly be finished soon.

I hate that testers that play this game are testing these SA's. I really hate that because they are doing so their findings will be publicly posted (as they should be). Soon everyone will not only know that SA's work but will know exactly how to make them work. This will be the end of "experimenting with build ideas". This should be a project for Bort and Catch or anyone that does not play this game so that the finding can be kept private and no advantage will be gained by doing so.

Originally posted by beenlurken
I hate that testers that play this game are testing these SA's. I really hate that because they are doing so their findings will be publicly posted (as they should be). Soon everyone will not only know that SA's work but will know exactly how to make them work. This will be the end of "experimenting with build ideas". This should be a project for Bort and Catch or anyone that does not play this game so that the finding can be kept private and no advantage will be gained by doing so.

the problem with only having 2 people doing it is it is an extremely extensive process and requires A LOT of time and effort. If Bort and catch were to do it alone, it would take YEARS to do all the necessary testing. And its not so much as to how to make them work, its whether or not they work as Bort/Catch want them to.

If I go with the description of how the testing will be performed by AngryAnton... how will you know if something isnt working because of the SA level or because the supporting attributes/VA's/etc arent high enough... how do you know which supporting attributes/VA's/etc should be pumped to activate the SA to even begin? That said, I assume Bort/Catch have given yall levels at which these SA's should work and yall are testing to see whether they do or not. Considering yall know this info, I assume we will too (its only fair). So if something works (why should it be changed... unless it works to well... we should expect it to take more for supporting variable but at least now we have 100% confirmation of what these variables are and it will be very easy to figure new levels for activation) it will stay the same and if something doesnt than I assume it will be buffed to do so at the levels they gave you. End result is that we will now know exactly how to build perfect dots. Just dont see how that is good for the game.

I guess damned if we do... damned if we dont...

Originally posted by beenlurken
If I go with the description of how the testing will be performed by AngryAnton... how will you know if something isnt working because of the SA level or because the supporting attributes/VA's/etc arent high enough... how do you know which supporting attributes/VA's/etc should be pumped to activate the SA to even begin? That said, I assume Bort/Catch have given yall levels at which these SA's should work and yall are testing to see whether they do or not. Considering yall know this info, I assume we will too (its only fair). So if something works (why should it be changed... unless it works to well... we should expect it to take more for supporting variable but at least now we have 100% confirmation of what these variables are and it will be very easy to figure new levels for activation) it will stay the same and if something doesnt than I assume it will be buffed to do so at the levels they gave you. End result is that we will now know exactly how to build perfect dots. Just dont see how that is good for the game.

I guess damned if we do... damned if we dont...

no, nobody was given levels at which the stuff should work at (to my knowledge). I'm fairly sure only bort and maybe catch know that, but the tests with 5, 10, and 15 in the SAs will be compared to the control (0 in the SAs) to see what/if any effect they have. Basically we're comparing what effect having the SA has in comparison to not having the SA. If there's no perceivable effect. For example, if having 15 in sticky hands doesn't increase catch percentage versus having 0 in it or if 15 in Lower the Shoulder doesn't increase BTKs, it'll be shown to catch/bort and they'll look into it. At least, to my knowledge that's how this test is going to be designed to work. Anton will have more information on this than I would but it seems obvious that Catch doesn't want people to know exactly when things start working since we're only doing testing at 5,10 and 15 and not 1-15.

Originally posted by SikoraP13 DTD
Originally posted by beenlurken

If I go with the description of how the testing will be performed by AngryAnton... how will you know if something isnt working because of the SA level or because the supporting attributes/VA's/etc arent high enough... how do you know which supporting attributes/VA's/etc should be pumped to activate the SA to even begin? That said, I assume Bort/Catch have given yall levels at which these SA's should work and yall are testing to see whether they do or not. Considering yall know this info, I assume we will too (its only fair). So if something works (why should it be changed... unless it works to well... we should expect it to take more for supporting variable but at least now we have 100% confirmation of what these variables are and it will be very easy to figure new levels for activation) it will stay the same and if something doesnt than I assume it will be buffed to do so at the levels they gave you. End result is that we will now know exactly how to build perfect dots. Just dont see how that is good for the game.

I guess damned if we do... damned if we dont...

no, nobody was given levels at which the stuff should work at (to my knowledge). I'm fairly sure only bort and maybe catch know that, but the tests with 5, 10, and 15 in the SAs will be compared to the control (0 in the SAs) to see what/if any effect they have. Basically we're comparing what effect having the SA has in comparison to not having the SA. If there's no perceivable effect. For example, if having 15 in sticky hands doesn't increase catch percentage versus having 0 in it or if 15 in Lower the Shoulder doesn't increase BTKs, it'll be shown to catch/bort and they'll look into it. At least, to my knowledge that's how this test is going to be designed to work. Anton will have more information on this than I would but it seems obvious that Catch doesn't want people to know exactly when things start working since we're only doing testing at 5,10 and 15 and not 1-15.

Yea but the difference between 5/10/15 sticky hands with 50 catching (and/or other factors) maybe minuscule (yall conclude not working) when there might be a huge difference between 5/10/15 sticky hands with 70 catching (and/or other factors). Just dont see how these tests will be worthwhile without have a determined base build that should produce results (performance a 5 with a vastly improved performance at 15).

Originally posted by beenlurken
Just dont see how these tests will be worthwhile without have a determined base build that should produce results (performance a 5 with a vastly improved performance at 15).

If i am not mistaken it has been stated by Bort in the past that SAs are independent of any attribute levels, unless stated otherwise in the description of said AI (as in "takes more Agility to pull off successfully"). In other words, it should not really matter at what level a certain skill is, you either see an effect from putting points into it or you won't. And this should hold true even for those like Juke that will become more efficient with a higher base skill, at the same base skill it should still fire more often the higher it is, regardless of how high the base skill is.

Aside from that there are the base builds for the players on the test teams which are used as a baseline, I doubt that Bort and Catch would have approved those if they were totally out of spec for what was intended in the first place.

so what youre saying is... you never tested them before you implemented them?

thats what i thought.

So basically you guys are just testing to see if the SA "fires" at appropriate rates for 5/10/15 and are not really concerned with its effectiveness (since there are so many influencing variables that influence its effectiveness). If that is all you guys are doing then that is really no big deal.

That said, a SA like Growl (example of a SA that many believe does not work) may fire at appropriate rates at 5/10/15 but the supporting attributes that really make it effective may need to be unreasonably high (or higher than intended by Bort) and it makes the whole point moot (you can have a SA fire on every play but if it doesnt produce results then it is worthless). This part of code needs to be checked too and should be done by those not playing the game.