User Pass
Home Sign Up Contact Log In
Forum > Test Server Discussion > Test Server Discussions > Current Sim Issue- SA's/VA's That Are Too Strong Or Too Weak
Page:
 
tet
offline
Link
 
this makes so much sense. in fact early on i figured this was happening already and was surprised it wasn't...

not only could it be applied to SAs/VAs on a test server, but to the sim in general analyzing real games, to test adjustments to the sim/builds.
 
Octowned
offline
Link
 
It would take a lot of servers and a full time statistician. That's probably a $200,000 investment just in testing, which I'm not sure GLB is ready to make.

But honestly.. You use an analyst/tester to get you to a steady state, move the game out of beta, market, and then use the analyst to produce real-time reports that mines the production databases to ensure the balance is maintained. You can move from testing to analyzing what drives player retention, boosting, optimal flex pricing, roster sizes, number of teams to introduce, where marketing campaigns are working, etc.

I'm not sure what type of staff GLB has, but I can't imagine this game getting very far without a position like this. Especially when it comes to finding the sim GLB needs.

(I'm not marketing myself, lol, I have a job . In fact, my current job works WITH sites like this, though )
 
taz20075
offline
Link
 
Originally posted by Octowned
It would take a lot of servers and a full time statistician. That's probably a $200,000 investment just in testing, which I'm not sure GLB is ready to make.

But honestly.. You use an analyst/tester to get you to a steady state and sell it to ESPN.


FYP.


Edited by taz20075 on Mar 20, 2010 18:06:08
 
Catch22
offline
Link
 
Originally posted by Octowned
Bort should hire a statistician IMO. OK, I am one, so I'm a bit biased, but..

Get rid of this "sim a test game and watch it twice" garbage. Do controlled experiments with thousands of sims. Think of the questions you can ACTUALLY answer that don't rely on 20 testers digging through pbp data.

Big picture analysis:
** what SAs/VAs actually work?
** what is the optimal build structures in certain situations?
** what plays are the most effective?

Sim change analysis:
** what effect does this VA/SA have on YPC, broken tackles, completion %, deflections, etc?
** how does performance decrease when the impact of energy is increased?
** what impact does roster depth have on team performance?


Etc., etc. One full time statistician with a few dedicated test servers with less boundaries (don't restrict testing to full games, etc.) could answer the most meaningful questions without actually watching the sims in full. That's the beauty of data analysis.


Hell, instead of guessing the right % to put on the bruiser VA, ask a question first:
What impact do I want 15 bruiser to have on YPC and broken tackles?
Answer it, 1 YPC and 20% more broken tackles, but with a variance on YPA under 3 (to control for too many breakout plays)
Then run 10,000 plays varying the % on the VA and see what gets the results you wanted. Uber guess and check.



Anyway, I don't mean to discredit the great work catch is doing, but until testing is done in large scale like this, the game will just continue to make mistakes and be unbalanced...


It's actually what I'm doing, but on a much smaller scale (we don't have a way to run more then one game a time on the test server, I have to wait for one to complete before starting a new one).
 
Guppy, Inc
offline
Link
 
is there no way to test an individual play? by testing full games, you cant be sure that any difference arent just products of the rng. from my observations, the rng at times, isnt very random and tends to clump. for simple proof, just look at the aeq shopping and how often most of the choices are similar in some way for any given token.
 
InRomoWeTrust
Lead Mod
offline
Link
 
Originally posted by Guppy, Inc
is there no way to test an individual play? by testing full games, you cant be sure that any difference arent just products of the rng. from my observations, the rng at times, isnt very random and tends to clump. for simple proof, just look at the aeq shopping and how often most of the choices are similar in some way for any given token.


We have the ability to turn off energy, morale, and the repeat play penalty. Tends to kill off a bunch of the 'randomness'.

Morale can very easily sway games against the same opponents to the point where you get very different results.
Edited by Mat McBriar on Mar 25, 2010 01:58:39
 
monsterkill
offline
Link
 
Originally posted by Guppy, Inc
is there no way to test an individual play?


was that not the first thing bort developed? if not ...

whatever, this whole approach of spamming test server games to reverse engineer the SAs is just absurd to me. does bort trust catch22 or not?

for example, the catch fake SA. bort should be able to use the CB's build, the WR build, their current energy and morale, and any other factors and tell catch22 'ok the wr had a 52% chance of a successful catch fake against that CB's relevant factors'. then you sit down and decide, 'is that what we want?' you do it for average match ups, extreme match ups. then adjust the formula to match your decisions. dead horse beating: writing the formula before making these decisions is terrible game design

why does catch22 have to run a ton of games, analyze a bunch of data just to arrive at a rough estimate of that 52% number before they can start deciding which way to balance it?

bort i am disappoint
 
tragula
title
offline
Link
 
Originally posted by monsterkill

for example, the catch fake SA. bort should be able to use the CB's build, the WR build, their current energy and morale, and any other factors and tell catch22 'ok the wr had a 52% chance of a successful catch fake against that CB's relevant factors'. then you sit down and decide, 'is that what we want?' you do it for average match ups, extreme match ups. then adjust the formula to match your decisions. dead horse beating: writing the formula before making these decisions is terrible game design


That sounds wrong. Bort should know to calculate the probability for a single roll. But there is dynamics between the rolls, the builds and the play being run that are hard to account for. Knowing the small details is not a guarantee to understand how the big system works.

I doubt that the sim is too complicated to figure out from the details. But you need to know how to analyze it, which is likely above the head of a "programmer".

 
Hikariu
offline
Link
 
I think bort can do a lot of this on his personal test server, just not the one catch/we have access to.
 
Catch22
offline
Link
 
Bort can test individual plays - we can't on the test server. I wish there was a more efficient way for me to test SA's and VA's... I really do. I'll talk to Bort and see if there is anything he can do to help me there.
 
tragula
title
offline
Link
 
Originally posted by Catch22
Bort can test individual plays - we can't on the test server. I wish there was a more efficient way for me to test SA's and VA's... I really do. I'll talk to Bort and see if there is anything he can do to help me there.


I vote to let catch have full access to Bort's personal test server.
(I hope my vote counts)
 
TrevJo
offline
Link
 
Originally posted by tuba_samurai
Originally posted by Mat McBriar

SAs that are probably too weak:

Turn the Shoulder (Quarterback)


Originally posted by Adderfist
Turn Shoulder is bad.


I'm kind of surprised this one wasn't mentioned in the post of potential game changes. How can you tell if Turn Shoulder activates? Does it ever?
 
taurran
offline
Link
 
Originally posted by Bort
- Clock Manager cut in half, reduced the time a little
- Hail Mary from 20 to 30 yds
- Hard Count back to 5%
- SYM between the tackles only
- DvG 3% to 2%
- Bull Rusher .5% across the board
- Heavyweight 1%

With this list, I could easily see everone going back to good ole attribute VAs from now on. At least they're becoming less and less "make or break" to a build.... this might not be a bad thing.
 
Sik Wit It
offline
Link
 
Originally posted by TrevJo
Originally posted by tuba_samurai

Originally posted by Mat McBriar


SAs that are probably too weak:

Turn the Shoulder (Quarterback)


Originally posted by Adderfist

Turn Shoulder is bad.


I'm kind of surprised this one wasn't mentioned in the post of potential game changes. How can you tell if Turn Shoulder activates? Does it ever?


It will look like the DE (or whoever) missed a tackle. They will shrink instead of getting the sack, then the QB will usually throw the ball. It won't say missed tackle or anything though.

I've seen it maybe 3 or 4 times.
 
Catch22
offline
Link
 
Thread locked by moderator.
 
Page:
 


You are not logged in. Please log in if you want to post a reply.