Warrior General Games

It would be nice to be able to watch a sim of the exact same builds and tactics back to back with one sim showing the current build of GLB and one showing the proposed sim.

The earlier that you show this to us, the sooner we can react and 'fix our builds."

Currently we get to see the effect of proposed changes only during the preseason. If they make the changes concurrently and they announce that they are happy with the changes, we can start making adjustments to our builds earlier or at least better plan for our players next season under the new sim.

Example:

This is how an agility NT interacted with the D-line last season:

http://goallineblitz.com/game/replay.pl?game_id=964200&pbp_id=15694449

This is how that same interaction looks this season:

http://goallineblitz.com/game/replay.pl?game_id=1064761&pbp_id=6850665

I wouldn't have built agility DTs to try and contain the pass if I had known what they were going to look like this season. This player has 63 Agility and is playing against an O-line that has 63 Strength and is run focused. He should be able to slip right around them like he did last season, but now when he wins the battles, he's not getting that slip through that he got last season.

peewee?

also, I can show you a zillion replays from last season that look exactly like the one you posted this season.. agility isn't the only part of the pass blocking rolls, you do need strength to win those battles.

Originally posted by sunshineduck
peewee?

also, I can show you a zillion replays from last season that look exactly like the one you posted this season.. agility isn't the only part of the pass blocking rolls, you do need strength to win those battles.

what? get the fuck outta here, that's crazy talk

Originally posted by sunshineduck
peewee?

also, I can show you a zillion replays from last season that look exactly like the one you posted this season.. agility isn't the only part of the pass blocking rolls, you do need strength to win those battles.

Doesn't answer my question or the basis of my post btw. Seeing EXACTLY how the new changes look compared to the current sim? You don't see the benefit of this, and instead are going to question my reasoning because the clips I showed you were from PWG.

Seriously?

I'm sorry - but I just don't see this as a practical option.

The sim is extremely complicated - and nothing is as simple as showing a replay or two to highlight a difference. There are just too many factors involved.

I certainly understand your frustration - it is very frustrating when a certain build performs very well one season - and has a significant drop-off the next. But - in this world of GLB with fairly consistent tweaks in the never ending quest for balance - it is what it is.

As I'm sure you know, all test server games are available for public viewing - I think keeping up with those will give you a much more meaningful sample size - and a much better, broader picture of what changes might be occurring.

Originally posted by TxSteve
I'm sorry - but I just don't see this as a practical option.

The sim is extremely complicated - and nothing is as simple as showing a replay or two to highlight a difference. There are just too many factors involved.

I certainly understand your frustration - it is very frustrating when a certain build performs very well one season - and has a significant drop-off the next. But - in this world of GLB with fairly consistent tweaks in the never ending quest for balance - it is what it is.

As I'm sure you know, all test server games are available for public viewing - I think keeping up with those will give you a much more meaningful sample size - and a much better, broader picture of what changes might be occurring.



So they can't load the rule set from the main server for the current season and sim four or five games and then load the new proposed rule set and sim four or five games to see what is different?

It seems like this should be EXACTLY what they are doing when rolling out new changes, no?

Originally posted by Time Trial
Originally posted by sunshineduck

peewee?

also, I can show you a zillion replays from last season that look exactly like the one you posted this season.. agility isn't the only part of the pass blocking rolls, you do need strength to win those battles.

Doesn't answer my question or the basis of my post btw. Seeing EXACTLY how the new changes look compared to the current sim? You don't see the benefit of this, and instead are going to question my reasoning because the clips I showed you were from PWG.

Seriously?

I don't think it's as simple as you'd like it to be.

There are too many variables that go in to determining what happens during a play, to compare a play pre changes to post changes accurately.

Even given the same 2 teams, player matchups, play call, AI, player level tactics, etc, you're still going to miss the RnG dice rolls on both sides of the ball which could be drastically different.

Replay examples are really only applicable to illustrate a new animation, i.e. last year there was no swim move, here's an example of what a successful swim looks like.

Originally posted by Time Trial
So they can't load the rule set from the main server for the current season and sim four or five games and then load the new proposed rule set and sim four or five games to see what is different?

It seems like this should be EXACTLY what they are doing when rolling out new changes, no?

I can tell that you're passionate about this (and irritated).

But that is exactly what is currently done. Everyone has access to the previous season's game - and everyone has access to the test server games - which are clearly labebelled things like "testing OL/DL interaction".

I think they ARE doing what you're asking - but it is in the form of the test server.

... Completely over your heads.


He wants a CONTROL. It's hard to see the changes when things like tactics have an effect on the game.

Originally posted by Adderfist
... Completely over your heads.


He wants a CONTROL. It's hard to see the changes when things like tactics have an effect on the game.

No, not really. I get it.
There's just
A) no mechanism in place for this
B) results aren't going to be conclusive due to other factors.

Originally posted by Adderfist
... Completely over your heads.


He wants a CONTROL. It's hard to see the changes when things like tactics have an effect on the game.

Yup.

I understand that a game using the same rules will look very different when simmed five or ten times, but you should be able to see some trends out of that data. But if all you are doing is looking at the sim under the new rules and not using the current sim using the same builds and tactics as a control, you have no idea what the new game is going to look like.

Let's use for example the new OL/DL interactions and the effect on the game. If you set your players tactics and AIs to simulate a normal game and ran that sim five or more times, you would get your normal (the current model). Then you change the rules to the proposed changes and run it five or more times, this is your experimental data.

Looking at the box scores you would be able to take an average of completion %s, hurries, sacks, etc. and see that the trend of the change is towards much more pressure on the QB. You could then watch the control games and the experimental games back to back and try and observe the effect on the games.

I realize that you can't look at one play in Sim A and one play in Sim B and say, "these two things aren't the same because of the new sim logic", but what you can do is run the experiment as described above and be able to make educated decisions about how the new logic affects the game.

The more times you run your control and the experiment, the more sound your data will be.

I think that you guys are doing a great job and that the work that catch22 is doing is very exciting, but I do think that without controls you are going to see these crazy roller-coaster reactions to changes in the sim and we'll never get out of Beta and on to something that we can call a finished product (and where changes come in the form of small patches rather than massive rule changes).

Originally posted by jamz
Originally posted by Adderfist

... Completely over your heads.
He wants a CONTROL. It's hard to see the changes when things like tactics have an effect on the game.
No, not really. I get it.
There's just
A) no mechanism in place for this
B) results aren't going to be conclusive due to other factors.

Without a "baseline" set of data, THEN any and all other tests will be unreliable and inaccurate. IF there isn't a mechanism in place, then this is the Perfect time to create one.

This is a basic concept of any software development and QA testing before release to a production environment.

Or you could just adapt.

Crazy concept, I know.

Originally posted by Deathblade
Or you could just adapt.

Crazy concept, I know.

Thanks Deathblade - are you able to at least offer any insight? I would expect that what he is asking for probably does go on - at least on Bort's machine?

Any constructive info you could share from your Test experience?

Controls aren't really possible on the test server. It'd have to be something Bort does on his dev box and I'm fairly confident that's not something he's going to do. I'm also not sure that many companies that let users have as much access to test server data as this one does (the main reason being that we use paying customers as testers and need to maintain transparency).

We're working on balancing/stabilizing the sim and the communications about what is being changed are going to be much clearer in the future as well as how changes are tested before being moved live. It might not be exactly what you want but give the new procedures some time to flesh out and see how they work.