View previous topic :: View next topic |
Author |
Message |
Doomkiller Posting Spree!
Joined: 02 Apr 2010
|
Posted: Sat Sep 25, 2010 6:53 pm Post subject: Weapon Damage Tests |
|
|
I decided to make a wad to investigate weapon damage. Thanks to the helpers: Konar, Titan, Chain, Devi.
Lets look at how SSG and sg damage decreases with distance:
These results are averages from pin point accurate shots at exact distances. Notice nothing changes much until about 256 units. At 512 units the sg and ssg apear to be equal as the narrowers sg spray starts to give it the edge. I want to look at swing shots also but they are very hard to test because of lack of consistancy.
Now lets turn to the BFG, specifically the blast zone and the angle its fires a. A few surprising facts here:
1. You dont need to be able to see the target for it to do damage. 50 degrees will do some damage but your fov stops at 45.
2. The angle you fire at has no effect on the damage except at 2 peak points.
3. The peak points are 0 degrees and 45 degrees. at 45 degrees only half the target is visible but the damage is nearly twice as much as at a small angle.
This is the damage at 640 units:
When investigating bFG damage at different distances it seems that between 0 and 512 units there is no difference. However, I did come across this strange bug which I would like someone to look at.
Demo:
http://www.fileden.com/files/2008/6/4/1944346/BFGbug_testing3.zdo
Wad:
http://www.fileden.com/files/2008/6/4/1944346/testing3.wad
Feel free to use the wad if anyone wants to do some experimenting of their own. |
|
Back to top |
|
|
Evolution Wicked Sick!
Joined: 06 Sep 2005
|
Posted: Sat Sep 25, 2010 7:30 pm Post subject: |
|
|
interesting! |
|
Back to top |
|
|
EarthQuake Wicked Sick!
Joined: 02 Apr 2004 Location: Athens, Ohio. Dieblieber gonna getcha!
|
Posted: Sat Sep 25, 2010 11:15 pm Post subject: |
|
|
Wow, good job on your findings.
Can you attempt the same tests under doom2.exe or Chocolate Doom? |
|
Back to top |
|
|
mikehail On a Rampage!
Joined: 24 Mar 2007
|
Posted: Sat Sep 25, 2010 11:21 pm Post subject: |
|
|
did I miss something? Seems like it could be important to know how much data was collected and to have access to that data if possible... |
|
Back to top |
|
|
Dopefish Dominating!
Joined: 18 Aug 2006 Location: From the shadows I come!
|
Posted: Sun Sep 26, 2010 12:34 pm Post subject: |
|
|
mikehail wrote: | did I miss something? Seems like it could be important to know how much data was collected and to have access to that data if possible... |
exactly..I find it strange ssg damage is lower than sg at 500 ups. Also it's funny the damage with sg is lower at 780 ups than at 900. Sounds like bs to me |
|
Back to top |
|
|
Kilgore Air Cavalry
Joined: 17 Jun 2003 Location: Up the river
|
Posted: Sun Sep 26, 2010 1:23 pm Post subject: |
|
|
It would help if we knew how many observations were used to obtain the averages.
A very interesting exercise would be to estimate not only the mean damage, but also the sigma for each data point and then plot the values of +/- 1.95 * sigma. It will give us some idea about the reliability of the numbers. |
|
Back to top |
|
|
Doomkiller Posting Spree!
Joined: 02 Apr 2010
|
Posted: Sun Sep 26, 2010 5:22 pm Post subject: |
|
|
The first 5 SSG measurements were averaged from 10 readings. The rest of the data was taken with 4 readings. The SG damage in particular was quite consistant.
Has anyone checked the demo yet? |
|
Back to top |
|
|
phenex2 Unstoppable!
Joined: 10 Jan 2008
|
Posted: Sun Sep 26, 2010 5:50 pm Post subject: |
|
|
Doomkiller wrote: |
Has anyone checked the demo yet? |
Yes i have, but have you checked doom2.exe yet? :p
i wouldnt like to investigate something that is normal doom behaviour.
If its different from doom2.exe we will check it out in detail.
And as kilgore said you should plot twice the standard deviation as well.
I would suggest to increase the number of samples too.
10 or 5 are not really enough to get a proper picture. |
|
Back to top |
|
|
Doomkiller Posting Spree!
Joined: 02 Apr 2010
|
Posted: Sun Sep 26, 2010 7:01 pm Post subject: |
|
|
In teh real worl doing 100 woudl be great but i just dont have the patience, do you? ;P
I'll do some more tonight (maybe 20) and then try doom2.exe |
|
Back to top |
|
|
EarthQuake Wicked Sick!
Joined: 02 Apr 2004 Location: Athens, Ohio. Dieblieber gonna getcha!
|
Posted: Sun Sep 26, 2010 8:34 pm Post subject: |
|
|
5 to 10? That's quite extremely low. If you want these results to mean anything, you need to drastically increase your number of trials. If this takes a lot of time to set up, maybe you could try using the map to quickly set up the scenario, putting players in their proper spots/angles/etc (with teleports) and maybe even using some DeHackEd or ACS to ease the testing and produce some reliable test results (except you can't rely on those in doom2.exe). |
|
Back to top |
|
|
Doomkiller Posting Spree!
Joined: 02 Apr 2010
|
Posted: Sun Sep 26, 2010 9:57 pm Post subject: |
|
|
Ive got a better idea. If someone could point me towards the code/algorithm that generates the damage i might be able to analyze it. Does zd have its own ssg damage calculator. If so kilgore do you mind if i take a look. If not does anyone know where i can find it? |
|
Back to top |
|
|
mikehail On a Rampage!
Joined: 24 Mar 2007
|
Posted: Mon Sep 27, 2010 2:34 am Post subject: |
|
|
these tests were done rather poorly it appears, scientific method is king!
Doomkiller wrote: | I decided to make a wad to investigate weapon damage. Thanks to the helpers: Konar, Titan, Chain, Devi.
Lets look at how SSG and sg damage decreases with distance:
These results are averages from pin point accurate shots at exact distances. Notice nothing changes much until about 256 units. At 512 units the sg and ssg apear to be equal as the narrowers sg spray starts to give it the edge. I want to look at swing shots also but they are very hard to test because of lack of consistancy.
|
First of all, there are probably a few terms that should be defined here. What is "pin point" accuracy? Is a pixel crosshair drawn upon a player? Is an aimbot used? Did some guy just "aim as best he could" at another player? To me, pinpoint accuracy would mean the single pixel on a player that is most probably the most damaging. Note, these pixels are quite possibly different with the use of mouselook although that is just an assumption! I also have no clue what pixel is the correct pixel to aim at, unless this term is defined differently. It might be nice to explain how this is going to be achieved as well, for replication purposes. Also note, finding out which pixel is the most damaging is probably its own set of tests in itself, for these purposes it would probably work well enough to just use the same pixel.
The SSG and SG should probably be defined for those of us who may be unfamiliar, or just to assure us we are all on the same page here. I assume the SSG has 20 pellets and can deal from 0-300 points of damage because each pellet can hit for 5-15 and 0-100% of the pellets can hit the target. That gives us 61 values of damage possible. The SG would be similar then, 7 pellets each for 5-15 damage points with a 0-100% hit ratio. That gives us 22 values of damage possible. Here I am assuming both of these weapons have spreads high enough that at some distance X, they can be aimed perfectly and still miss, this might not be the case but I would be foolish to assume otherwise.
http://doom.wikia.com/wiki/Super_shotgun#Data
http://doom.wikia.com/wiki/Shotgun#Data
I am not going to define every statistics term here as they are well established, doom terms are much different and as such I think it is necessary they be given definition.
I think that is it for terms (screw you and screw your BFG).
While we are being all science-y, why don't we write out a theory and hypothesis? The SG seems to outdamage the SSG.
Because of the characteristics of the SG, it will deal more damage than the SSG at distances greater than 800 units when shot with "pin point" accuracy.
OR
Because of the characteristics of the SG, it will deal a lesser or equal amount of damage than the SSG at distances greater than 800 units when shot with "pin point" accuracy.
It is going to get fun for a while here and I apologize if I make some mistakes, gonna see what I can do here.
So I am gonna say there is 930 distinct possibilities in this scenario regarding possible damage outputs and hitting any number of pellets from 0-20 for the SSG. This means our sample size should be 273 samples per distance reading for the SSG.
Sample Size formula
n=(z^2pq)/(c^2)
n=sample size
z=z value
p=variability, .5 used for max variability
q=1-p
c =confidence interval, expressed as decimal
n={(1.96^2)(.5)(1-.5)}/(.05^2)
n=384.16
it is a relatively small population so to make it a little nicer we can use the finite population correction formula which states that:
If the population is small then the sample size can be reduced slightly. This is because a given sample size provides proportionately more information for a small population than for a large population.
n=(n0)/1+({n0-1}/N)
n=new sample size
n0=old sample size
N=population
n=(385)/1+({385-1}/930)
n=272.49
There is good news though! The SG only has 7 pellets and thus has a much smaller pool of possibilities. Only 93 tests would be needed to draw an accurate picture about SG damage spreads!
n=(z^2pq)/(c^2)
n=({1.96^2)(.5)(1-.5)}/(.05^2)
n=384.16
n=(n0)/1+({n0-1}/N)
n=(385)/1+({385-1}/122)
N=92.83
I am sure I made some mistakes in all this by using population based formulas instead of more advanced stuff, but I was quite lazy and this is still much better. Even at worst, this will be a much more reliable test than what was initially done, taking 10 samples is just ridiculous, there are more than 6 times as many possible outcomes from a single shot!
Also, that graph is pretty terrible, it is difficult to find out where you actually tested from because it is just a random interval that doesn't appear to coincide with a single set of data collected. I would suggest a much more accurate graph with a better scale, perhaps even a scatter plot with a curvilinear regression? You could even set it up to graph the confidence interval along with the regression to show your margin of error relative to the data collection and be awesome! The biggest job here is the data collection.
With all that being said, it is a very cool idea that you have started with, I hope you will continue to see it through. I would not mind helping if you were thinking of such an undertaking. The main problem here is your initial experiment was not very robust at all, data collection was minimal, nothing was defined, and the analysis was relatively poor. You could also set up a similar BFG experiment relatively easily but I have already wasted enough time here without knowing whether or not you will continue on with this endeavor! |
|
Back to top |
|
|
Kilgore Air Cavalry
Joined: 17 Jun 2003 Location: Up the river
|
Posted: Mon Sep 27, 2010 3:42 am Post subject: |
|
|
mikehail: are you trying to impress us all with that mumbo jumbo? this is a very simple test to conduct:- You have a person standing stlll at some predefined distance. You aim straight at him with the weapon you're testing (that's a good use of the crosshair). You shoot: then you record how much damage he received.
- You repeat the above 30 times or more (check out the term "Central Limit Theorem" if you want to know why 30). From that you can compute mean and sigma of the damage values.
That's it and you can make the above graph very well. What the heck are you talkiing about sample size, etc? will you do it any better or any simpler than what I described? just compare your explanation's length to mine. |
|
Back to top |
|
|
mikehail On a Rampage!
Joined: 24 Mar 2007
|
Posted: Mon Sep 27, 2010 4:47 am Post subject: |
|
|
I was not trying to impress anybody, except to hopefully goad DK into setting up something a little more useful. I thought it was rather important that he could see the math I did so he could agree/disagree with what I suggested. It sure is a simple test, but even the simplest of tests will be ruined through poor methods. For instance, you say to aim straight at him. What does that mean? What if your idea of "straight at him" is different than his idea? If I aim at his shoes does that count? If you are going to do an experiment why half-ass it? I was hoping he would explain some of the terms he was tossing around so that we could better understand whats going on.
I definitely agree though, I was pretty sure that I picked a rather irrational method to approach the problem at hand but that is how we learn. It was a fun little exercise to play with, even if I did it in a rather stupid way. The numbers I got would work fine, if the data collection might be considered a little overzealous. Nothing was ever hurt by collecting extra samples or collecting them in a rigorous fashion! |
|
Back to top |
|
|
rygrass God like!
Joined: 06 Nov 2005 Location: Aussie Clan:eV
|
Posted: Mon Sep 27, 2010 5:20 am Post subject: |
|
|
I tend to agree with Mike here. This needs to be more defined as if i was to do this same test under the same conditions you would expect me to get the same results??.
I'm not getting at you Doomkiller, just seems this test can be done a bit better. Its great that have gathered some of this information. |
|
Back to top |
|
|
DX-Chain God like!
Joined: 24 Jan 2006
|
Posted: Mon Sep 27, 2010 5:34 am Post subject: |
|
|
The WAD used for testing teleported the "shooter" to a teleport destination facing straight ahead of the "target" player. With each piece of data collected, the shooter would only press the fire key after putting himself in place for each amount of distance to be tested.
So no, "pin point accuracy" in DK's case means no freelook or shooting at anyone's feet. I do agree that there should have been more data taken but this is a nice thing to get more insight on the qualities of the weapons |
|
Back to top |
|
|
Doomkiller Posting Spree!
Joined: 02 Apr 2010
|
Posted: Mon Sep 27, 2010 9:22 am Post subject: |
|
|
Yes mike, you should be looking at the wad at least if you have issues with the accuracy of the results.
I never said that these results were definitive. 5 times is not enough? Yesterday I did 100 shots and then thought "now i only have to do this 1500 times more" (which is a euphemism for gave up). Perhaps someone else would like to use the wad to collect 1600 results and then type them all into excel. Even if we did 30 at each distance thats 480.
So at them moment I'm looking at the code. Unfortuantly we have 2 random variables: The angle offset and the 5-15 damage for each bullet. Averaging this out would be beyond me so I figured the best way to do this would be to run a computer simulation. I'll let you knwo if I have any success.
BTW: does anyone know how many "fine angles" there are in 1 full turn? |
|
Back to top |
|
|
Kilgore Air Cavalry
Joined: 17 Jun 2003 Location: Up the river
|
Posted: Mon Sep 27, 2010 11:19 am Post subject: |
|
|
Cut the crap mikehail: your population, sample sizes, confidence interval, etc stuff does absolutely nothing to solve the problem. The only real issues here are the number of observations and accuracy. You didn't resolve either with all that talk. Besides, the accuracy issue has been resolved by the design of the map. You'd realized that if you had bothered to test it rather than posting.
mikehail wrote: | Also, that graph is pretty terrible ... | Doomkiller did something nice (which can be improved of course) and you come out with such a rude statement without having contributed (or even cheched) at all. Nice going. |
|
Back to top |
|
|
dewww God like!
Joined: 29 Dec 2006
|
Posted: Mon Sep 27, 2010 1:32 pm Post subject: |
|
|
Kilgore wrote: | mikehail wrote: | Also, that graph is pretty terrible ... | Doomkiller did something nice (which can be improved of course) and you come out with such a rude statement without having contributed (or even cheched) at all. Nice going. |
eh, well.. i see mike trying to contribute and getting it wrong as well. :) no need to be so cold to him, imo. i know he spent a couple of hours studying statistics before that post. :) dk's intentions are great, but there's definitely some shortcomings.
the graph is only a rough approximation. the marked positions should be connected to exact values on both axes. dk's post suggests 128 units increment in the distance graph, but the 2 farthest spawns are both 8 units off (904 and 1032 units). the graphs don't indicate any values so i don't know if this is deliberate or an error. the excel sheet with measured data would help as well.
by the way, this also explains the mysterious 'bfg bug' since bfg rays only work up to 1024 units.
EDIT: ^disregard that, i am stupid and need to doublecheck things. dk is using the teleporter at 904 units. sorry!
another thing is the approximation used to 'connect the dots'. i'd say don't do it, use collumn-type histograms. for example doom wiki states the minimal damage for bfg is 49, dealt by one tracer ray hitting the target. the approximation between 50 and 55 degrees suggests some interpolation that doesn't happen (in doom2.exe).
by the way, this also explains the values in dk's angle graph: 2 tracers hit at 0, 1 tracer at other angles, 2 or 3 at 45 degrees (which i find rather weird). the damage variations for the 1 tracer values are caused by prng throwing it's dice.
anyways, let's not be condescending to each other and try to get the damage test right. i'm sure these data could be helpful in the long run!
Last edited by dewww on Mon Sep 27, 2010 4:59 pm; edited 1 time in total |
|
Back to top |
|
|
Cybershark Spamming!
Joined: 05 Jan 2005 Location: off the grid, but still fighting for the users!
|
Posted: Mon Sep 27, 2010 2:01 pm Post subject: |
|
|
Lol swingshots. Didn't someone like Quasar once look into the (Zdoom) code and determine that there was absolutely no evidence for the existence of them? |
|
Back to top |
|
|
|