User Pass
Home Sign Up Contact Log In
Forum > Discuss GLB Issues With Catch22 > NOT A BUG - CLOSED > Problem with Package Auto Adjust - NOT A BUG - CLOSED
Page:
 
climberpete
offline
Link
 
There is a problem with either:
Auto adjust turning on in packages where it is turned off.
The code surrounding play selection in a package is not truly random.

Preface: I am a tester with test server access and willing to carry out more tests if this is deemed to be a bug.

Data: I set my AI up in the off season and haven't touched it or looked at the results until today. I created 10 new packages. Two for each new formation, one with all the pass plays and one with all the rush plays. I also created a package that had all the new pass plays for the old formations and same for the new rush plays for the old formations. This gave me 10 packages. I set up an AI that had 1 input, fire always, and each of the ten packages was set as an output. Auto adjust was turned off both in the packages and in the AI. My team has now played 7 games and the AI side of things seems to be working perfectly:

Package - Number of times called
Old Rush - 68
Old Pass - 81
Big I Rush - 71
Big I Pass - 80
SBB Rush - 73
SBB Pass - 78
Trips Rush - 83
Trips Pass - 81
Spread Rush - 65
Spread Pass - 72

The mean of this dataset is 75.2 with a standard deviation of 6.2. The deviation is 12% of the mean. This is a reasonable random data set.

But when the individual packages are considered the distribution breaks down, rather than list all the packages (all show this problem) I'll break down one in detail:
Spread Pass:
Play - Number of times called
Diamonds 11
4WR Slants 7
Double Cross 6
Ins and Outs 6
WR Drags 6
Strong Overload 6
HB Clearout 5
WR1 Clearout 5
4 Wide Hooks 5
HB Screen Strong 4
4 WR Unders 3
WR Screen Strong 3
4WR Outs 2
HB Screen Weak 1
Weak Overload 1
WR Screen Weak 1

The mean of this dataset is 4.5 with a standard deviation of 2.65. This is a staggering 58% of the mean. This is not a random dataset. Something is breaking down in the play selection with the packages.

Feel free to look over the team: http://goallineblitz.com/game/team.pl?team_id=1386
And I'll send over GM invites to anyone who wants to see the AI and packages. All 10 packages in this test fail to have a normal random distribution of plays called.
 
Mightyhalo
offline
Link
 
I think it's just too many options. That is a huge number of plays in the package(16) with a chance to fire.

9 of the 16 were ran between 4 and 7 times.

 
climberpete
offline
Link
 
Originally posted by Mightyhalo
I think it's just too many options. That is a huge number of plays in the package(16) with a chance to fire.

9 of the 16 were ran between 4 and 7 times.



There are just too many outliers though. And every package has this disparity, there is always a play or two that get called more than double the mean or a play or two that almost never gets called. It's statistically improbable that a random dataset would have so many outliers.
 
drewsky
offline
Link
 
Check out what's being called more often in that package. The long/med pass plays are getting called more than the short.

Spread Pass:
Play - Number of times called
Med WR = Diamonds 11
Med WR = 4WR Slants 7
Med WR = Double Cross 6
Med WR = Ins and Outs 6
Med WR = WR Drags 6
Med WR = Strong Overload 6
Med HB = HB Clearout 5
Med WR = WR1 Clearout 5
Med WR = 4 Wide Hooks 5
Short HB = HB Screen Strong 4
Med WR = 4 WR Unders 3
Short WR = WR Screen Strong 3
Short WR = 4WR Outs 2
Short HB = HB Screen Weak 1
Med WR = Weak Overload 1
Short WR = WR Screen Weak 1
 
climberpete
offline
Link
 
Originally posted by drewsky
Check out what's being called more often in that package. The long/med pass plays are getting called more than the short.

Spread Pass:
Play - Number of times called
Med WR = Diamonds 11
Med WR = 4WR Slants 7
Med WR = Double Cross 6
Med WR = Ins and Outs 6
Med WR = WR Drags 6
Med WR = Strong Overload 6
Med HB = HB Clearout 5
Med WR = WR1 Clearout 5
Med WR = 4 Wide Hooks 5
Short HB = HB Screen Strong 4
Med WR = 4 WR Unders 3
Short WR = WR Screen Strong 3
Short WR = 4WR Outs 2
Short HB = HB Screen Weak 1
Med WR = Weak Overload 1
Short WR = WR Screen Weak 1


Interesting, I hadn't noticed that. I checked the basic settings of the team
Short - 35%
Med - 35%
Long - 30%

Pass to
WR - 50%
TE - 25%
FB - 15%
HB - 10%
 
drewsky
offline
Link
 
Package Breakdown
63% - 10 Med WR Plays -
19% - 3 Short WR Plays -
13% - 2 Short HB Plays -
06% - 1 Med HB Play -

Plays Called
78% - 56 Med WR Plays
08% - 6 Short WR Plays
07% - 5 Short HB Plays
07% - 5 Med HB Play


 
drewsky
offline
Link
 
There might be some sim bias to go with longer plays on 3rd and long / end of half / losing.
 
climberpete
offline
Link
 
It definitely could be losing, I don't have a full roster or a real AI so I lost most games by 100+

Plus I always went for it on 4th so maybe it's forcing long passes more often on 4th down.
Edited by climberpete on Nov 8, 2011 11:14:35
 
Catch22
offline
Link
 
Way too small of a sample size to come to any conclusions that this is a bug. You're talking less than 100 instances within each package. With a RNG, there is of course going to be a disparity especially when dealing with a small sample size. It doesn't look at how often the play has been called in the past, it merely looks at the probability the play should be called for this one specific instance. If you want to prevent a disparity such as this, your best bet is to have less plays within the package.

I'll leave this up for now but my initial (and probable) inclination is that this is not a bug.
 
climberpete
offline
Link
 
Originally posted by Catch22
Way too small of a sample size to come to any conclusions that this is a bug. You're talking less than 100 instances within each package. With a RNG, there is of course going to be a disparity especially when dealing with a small sample size. It doesn't look at how often the play has been called in the past, it merely looks at the probability the play should be called for this one specific instance. If you want to prevent a disparity such as this, your best bet is to have less plays within the package.

I'll leave this up for now but my initial (and probable) inclination is that this is not a bug.


Like I said, I'm a tester with test server access. I could run through a bunch of games to broaden the sample size. It would be a fairly simple test.
Run 10 games with 5 plays directly in the AI - control group
Run 10 games with 5 plays in a package - test group

At approximately 200 plays per game that would give two sets of 2000 plays called.

By picking two teams of differing levels this might also give us an indication of if long plays are really being called more frequently when losing.
 
Dr. E
online
Link
 
Is this the order they are in the package also?

Spread Pass:
Play - Number of times called
Med WR = Diamonds 11
Med WR = 4WR Slants 7
Med WR = Double Cross 6
Med WR = Ins and Outs 6
Med WR = WR Drags 6
Med WR = Strong Overload 6
Med HB = HB Clearout 5
Med WR = WR1 Clearout 5
Med WR = 4 Wide Hooks 5
Short HB = HB Screen Strong 4
Med WR = 4 WR Unders 3
Short WR = WR Screen Strong 3
Short WR = 4WR Outs 2
Short HB = HB Screen Weak 1
Med WR = Weak Overload 1
Short WR = WR Screen Weak 1
 
alindyl
offline
Link
 
Was thinking about that too, if the way plays are selected is done by starting at the first play and "rolling" a % chance for each play one at a time until one hits a successful chance.

I'm not a statistician but it would seem that if it does that instead of taking all the plays, assigning them X number of places out of 100% like an old D&D look up table, could potentially cause- at least in a smaller analysis to show some bias toward plays that get the first chance over ones at the bottom. Ie Diamonds always rolls first, WR screen weak always rolls last.

Just speculation though, if this is the case that the list above is the way they are ordered in the package.
 
climberpete
offline
Link
 
In the package they are in the following order:

Med HB = HB Clearout 5
Med WR = 4 Wide Hooks 5
Med WR = Diamonds 11
Med WR = Ins and Outs 6
Short HB = HB Screen Strong 4
Med WR = WR1 Clearout 5
Med WR = Weak Overload 1
Med WR = 4 WR Unders 3
Med WR = WR Drags 6
Med WR = Double Cross 6
Med WR = Strong Overload 6
Med WR = 4WR Slants 7
Short HB = HB Screen Weak 1
Short WR = 4WR Outs 2
Short WR = WR Screen Weak 1
Short WR = WR Screen Strong 3
 
alindyl
offline
Link
 
ok so much for that as a possibility.
 
Dr. E
online
Link
 
Originally posted by climberpete
In the package they are in the following order:

Med HB = HB Clearout 5
Med WR = 4 Wide Hooks 5
Med WR = Diamonds 11
Med WR = Ins and Outs 6
Short HB = HB Screen Strong 4
Med WR = WR1 Clearout 5
Med WR = Weak Overload 1
Med WR = 4 WR Unders 3
Med WR = WR Drags 6
Med WR = Double Cross 6
Med WR = Strong Overload 6
Med WR = 4WR Slants 7
Short HB = HB Screen Weak 1
Short WR = 4WR Outs 2
Short WR = WR Screen Weak 1
Short WR = WR Screen Strong 3


Looks about right then. Try it with dice. You will get results that look similar.
 
Page:
 


You are not logged in. Please log in if you want to post a reply.