Difference between revisions of "VROC Technical Notes"

From Grand Prix Legends Wiki
Jump to: navigation, search
(What to Do)
(Collision Prediction in Internet Racing)
Line 89: Line 89:
 
Ok, here's the scenario: I hosted via VROC, and had a full 20 cars, with only one player having disco problems, and I think it was on his end.
 
Ok, here's the scenario: I hosted via VROC, and had a full 20 cars, with only one player having disco problems, and I think it was on his end.
  
[Here Doc mentioned an impossibly short lap time set in qualifying. In the Clock Smashes section I attempt to explain why this and other problems mentioned above occur. - Ed.]
+
<span style="background:#00FF00">[Here Doc mentioned an impossibly short lap time set in qualifying. In the Clock Smashes section I attempt to explain why this and other problems mentioned above occur. - Ed.]<span>
  
 
Everything was fine up to the start...and even then it was working well. I usually see (if you're underpowered in the CPU department) folks getting dumped when the drive on the host system hits their green button to place their car on the grid. This didn't happen, and we had a full 19 cars at the start, and I was on grid showing 28 fps. At the drop of the green flag, the frame rate actually increased to around 32~33, and by one second into the race I was back at a pegged 36. (Too cool at 800x600 with most everything checked and bias at 1/3 from the left, mirrors to cars and track w/textures on.)
 
Everything was fine up to the start...and even then it was working well. I usually see (if you're underpowered in the CPU department) folks getting dumped when the drive on the host system hits their green button to place their car on the grid. This didn't happen, and we had a full 19 cars at the start, and I was on grid showing 28 fps. At the drop of the green flag, the frame rate actually increased to around 32~33, and by one second into the race I was back at a pegged 36. (Too cool at 800x600 with most everything checked and bias at 1/3 from the left, mirrors to cars and track w/textures on.)

Revision as of 16:23, 17 April 2009

Problems Connecting

Question: When I get to GPL's multiplayer screen, I only get COM ports (3 and 4) as the connection alternatives, not the "TCP/IP node". Are there any core.ini files that would work in this situation?

Most likely you need to have GPL look up the IP address of your machine using an alternate method.

DSL and cable connections often need to use an alternate IP address lookup method with GPL. This also happens with AOL dialup connections and may happen with any connection in which the ISP uses DHCP to assign the customers' IP addresses.

Using a text editor such as Notepad, simply create a file called core.ini in your main GPL folder (if you installed GPL in the default location, this will be C:\SIERRA\gpl). Then add these lines to that file:

[ Communications ]
alternate_ip_addr_lookup = 1  ; Find IP addresses another way

There are details about this parameter in GPL's readme.txt file. You'll find this in your main GPL folder.

Question: Sometimes when I connect to a server, I get to the Weekend screen in GPL but there's no green button, and then I get kicked back to the Multiplayer screen.

This is a known problem in GPL. Usually you can simply try to connect again, and the connection will be made properly.

Note: on the Track Selection screen, after you connect, the Practice Remaining field will first show the initial number specified by the host (usually a round number like 15:00, or 30:00) and then changes to 0:00. When the initial handshaking is complete this number will switch to the time actually remaining, and start counting down. It's a good idea to wait until the number starts counting down before hitting the green button to advance to the Weekend screen.

If the Practice remaining doesn't start counting down within about 30 seconds, either the handshake has failed, and you'll get kicked back to the Multiplayer screen, or practice is over, and you can advance to the Weekend screen. If practice is over, you won't get a green button at the Weekend screen. In this case, simply wait for the session to switch to race, and then advance to the grid.

If you fail to connect consistently to a certain server, try another server. If you fail to connect to many servers, you may have a problem with your Internet connection.

Problems When Racing

Problems you man encounter when racing online include disconnects, reduced frame rate, frame stuttering, warping, winking out, screen flashes, and giant warps. These problems can usually be resolved by taking a few simple steps. This section should give you the background to do so.

If you've raced on VROC for long, you'll have noticed a number of issues that seem to affect online racing. These include:

  1. Some people disconnect frequently, and sometimes when such people join a session, other people get disconnected.
  2. Sometimes people do impossibly fast lap times, like one or two seconds or less.
  3. Sometimes people get disqualified for going backwards, or get a black flag for cutting the course, even when they haven't.
  4. Sometimes other cars disappear and reappear a large distance away. If you're a client, this may be associated with an abrupt flash of the screen, and a strange interruption in the sound of your car. Sometimes the other players' cars make strange warbles in their sound too.
  5. When races start, if there is a collision, suddenly cars start disappearing and reappearing all over the place, and crashes seem to be crazily violent.
  6. Sometimes the sim pauses right in the middle of driving, or runs horribly slowly, or does other nasty things related to system memory.

Below are suggestions about what you can do to prevent these things, followed by a discussion which explains why they happen. Following that is an explanation of how to save longer replays on machines with 64 mb of memory.

What to Do

To address problems 1 through 4 above, read Alison's GPL Online FAQ and follow my advice! If you don't have time to read the entire FAQ, here's a synopsis:

  • Please check if you have the latest GPL version installed. Latest GPL version 1.2.0.2 which can be obtained by using the All-In-One Patcher 2.3, using the GPLPS Installer which is recommended or apply all the official patches.
  • When hosting, use VROC's recommended Bandwidth Settings.
  • If you can, switch to a high speed digital Internet connection, such as cable modem, ADSL, or ISDN (preferably not using the serial port!) If that's not possible, please seriously consider a USB modem or an Internet Gaming Modem!
  • Adjust your graphics and sound settings in the sim's Options, and/or upgrade your hardware, so you get 36 fps all the time when you are racing online! A digital sound card can make a big difference here.
  • If you have a Winmodem, discard it immediately in favor of a real modem! Winmodems lack certain critical components, instead offloading more communications workload onto the CPU - exactly what you don't want for online racing!
  • Only join races on the VROC race list that show ping times consistently below 500-600 ms, and preferably below 300 ms.
  • If you rarely or never see races which meet the above requirement, tune your DUN and modem settings (see Alison's GPL Online FAQ and Doc Wynne's advice about setting MTU size)
  • Try new ISP's until you get good pings.

To avoid collisions at the start, do the following:

  • Practice standing starts offline until you can get a good start, keeping the car going dead straight, every time. You should be able to consistently make it to the first turn without losing more than two or three positions to AI cars in single player races.
  • Drive very carefully at the start, and make every effort to avoid a collision. Remember that the information you are getting about the cars ahead of and around you may be almost a second old, so leave plenty of room ahead and to the side.
  • Listen for other cars around you at the start.
  • If you have appropriate buttons on your controller, assign them to your sim's Look Left/Look Right functions, and use them!

To avoid pauses and other memory-related issues, see Doc Wynne's advice about Windows 95 memory on the next page.

Changing Replay Length in GPL

I'm not sure who wrote this but it sounds like it came from someone at Papyrus:

We use a conservative heuristic to determine how large a replay buffer to allocate to reduce the likelihood of paging during gameplay. GPL will always allocate a replay buffer of at least 2M, however, even if the heuristic says that's too much. On a 64M machine, you'll pretty much always be in this 2M zone. You're welcome to experiment with larger replay buffers, but I found that even doubling the buffer to 4M on a 64M machine lead to significant paging.

Play with this at your own risk.

In the same directory as the gpl.exe program, use Notepad (or another plain text editor) to create a text file named core.ini In this file, add the following two lines:

    [ Replay ]
    replayMemoryOverride = X

where X is the number of Kbytes of memory you want to dedicate to the replay buffer.

Several people have reported increasing memory for replays with no ill effects. However, keep in mind Doc Wynne's comments on memory allocation in Windows on the next page.

Collision Prediction in Internet Racing

Doc Wynne, who hosts races on VROC on his high speed connection, has written an excellent and very detailed discussion of the problems of predicting where cars will be after a collision in a high latency situation, such as on the Internet, and the reasons why a crash at the start of an online race wreaks such havoc.

Note that this article refers to GPL 1.0. An enhancement in GPL 1.2 significantly reduces the collision prediciton problem by making GPL's cars "softer" when latency is high, thus reducing the risk of overly violent collisions. Nevertheless, Doc's comments are still relevant, although the problem iw now much less severe.

Comments by Doc are in green; my responses are in black.

Ok, here's the scenario: I hosted via VROC, and had a full 20 cars, with only one player having disco problems, and I think it was on his end.

[Here Doc mentioned an impossibly short lap time set in qualifying. In the Clock Smashes section I attempt to explain why this and other problems mentioned above occur. - Ed.]

Everything was fine up to the start...and even then it was working well. I usually see (if you're underpowered in the CPU department) folks getting dumped when the drive on the host system hits their green button to place their car on the grid. This didn't happen, and we had a full 19 cars at the start, and I was on grid showing 28 fps. At the drop of the green flag, the frame rate actually increased to around 32~33, and by one second into the race I was back at a pegged 36. (Too cool at 800x600 with most everything checked and bias at 1/3 from the left, mirrors to cars and track w/textures on.)

Then the carnage began. One car in the back row ran into the car in the next row up, and pushed them into me...now at this point the cars aren't going terribly fast, but if flipped me end over end several times. Of course, I landed in front of more cars, and the pileup began. A full 30 seconds had passed before I was able to Shift-R my car to upright and try to chase the pack down.

Unknown to me, most of the middle of the pack had crashed in Curva Grande up ahead, and there were cars everywhere as I rounded the corner. I managed to brake and couldn't believe what I was seeing! I was driving in 1st, trying to weave my way through all the cars, but they kept warping into and out of view...you'd see a "hole" to try to use, and as soon as you went for it, a car would pop into view in front of you. I finally got through it all, but not without further mishaps...my first lap time due to all the wrecks was like 2:48! Needless to say, I was lapped by the 5th lap and didn't fare too well.

But I did save the whole race in replay and got to looking at it last night after it was all over. It looked like suspicions confirmed as to the collisions....the warping was caused by the collisions! It seems to me that the host is doing nothing more than collision detection, and sends a packet out to all player's computers telling them that (in effect, probably greatly simplified here) "car #xx just hit car ##yy in zzz location" and then everyone's own GPL does its own interpretation of the physics involved and models it accordingly. However, (and here's the catch) even with a Latency of 0.120, the cars actually involved in the crash don't send their actual positional data back to the host and then on to the rest of the pack until almost a 1/4 of a second later!

Assuming latency of .120, the crash occurs, and the host sees a collision in the car's positional data and says "there's a crash between #xx and #yy" -- the client systems don't get this message until at least .060 sec. after it's happened and they attempt to model it their way. In the meantime, the two cars actually involved have GPL model the crash from their perspective and as soon as the car's position is fixed in 3-D space by their respective copies of GPL, they send their new positional data back to the host - at least another .060 sec. assuming the good connect. The host receives this new info and passes it along to the other players (another .060 sec. and all this is assuming -incorrectly - that it takes zero time to process the info and resend it on both ends), who already have supposedly "modeled" the crash, but now here it is (via latency) arriving another .180 sec. later..and guess what? The cars aren't where your copy of GPL predicted they would be, so your copy of GPL just moves the wrecking cars to where they are as reported by the host from the data supplied by the wrecking cars.....instant warping! Of course, at some place like Monza where the cars are moving along pretty fast, the margin for error between where the car was predicted to be and where it actually reports itself as being is HUGE!

Add to the delay of the above the time it takes each player's copy of GPL to process the incoming data and respond, as well as if the data is produced in time to make the next available packet outbound to the host (core.ini settings):

If default is every 2 ticks or every 1/16 sec. (.0625), and MOST drivers are using the core.ini they d/l somewhere that updates every 3 ticks (.09375 sec.), the average delay in getting the packet out with accurate car position info. would be around half that, so add another 0.046 sec. to the lag.

Timeline:

  • Host detect coll. = 0.00
  • Host sends packet to players = 0.046 sec. (avg. core.ini delay)
  • Travel time (1/2 latency) = 0.060 sec. - total delay now 0.106 sec.
  • Player's computers figure physics, position cars. =0.000 (not right but instantly will be used for assumption) Player's computers send packet w/position back = 0.046 sec.
  • (avg.) total delay now 0.152 sec.
  • Travel time to host = 0.060 sec. - total delay now 0.212 sec.
  • Host resends to players = 0.046 sec. - total delay now 0.258 sec.
  • Travel time to players = 0.060 sec. - total delay now 0.318 sec.

Now, assuming (incorrectly) that figuring the physics of the car takes no time whatsoever, we still have a total delay in updating car positions of every 0.318 sec. (average.) after the actual collision was detected! Thereafter, the car physics would be updated a bit quicker, since the collision detection delay isn't calculated anymore and would equal the travel times plus packet delays: 0.212 sec. avg. (after all, no one knew there was a collision until their computer were told there was one!)

Best numbers would be if the packet is calculated and ready at the exact instant that the next packet is due to be sent: 0.060 times three trips = 0.180 sec. from initial collision detection, and updates on car position every 0.0180 sec. thereafter.

Worst numbers would be the max. delay in the way the packets are sent out - the new packet with the post-collision car position was ready just as the previous packet was sent, so we wait the full time for the next packet: 0.456 sec. from collision before first update, and updates every 0.364 sec. thereafter.

Considering this data can (at best in an online setting) be updated at only around .2 sec. intervals with almost perfect connects, is it any wonder any crash in a pack would result in a multi car pileup even with careful drivers? Think of how slow updating would be possible if everyone was connected via modem with a latency of around .250! Accurate car position updates then would take place less than 2 times per second, turning a close pack of cars into a wheeled version of billiards. You would think the whole pack is warping or the host is hosed, but it's merely GPL making assumptions and updating with facts whenever it can get them. Throughout the whole mess of this massive crash, I was still getting 36 fps! On another note, I followed all of the cars post crash for the rest of the race to see if there was one who was warping badly, and couldn't find one...everyone had pretty well warp-free connects.

Another thing to think of is how GPL probably models the physics of the crash as to how hard you were hit...relative velocities and all that. If a car just appears (due to packet delay as shown above) to be inside the envelope of your car, how does GPL look at that? I'm thinking GPL may have a max. limit on relative velocity, but since the car appeared from virtually nowhere, GPL still figures it's moving damned fast, and figures collision physics accordingly. This explains why some of the really low speed collisions between car result in violent flips and whatnot.

Now, if you take all of the above, knowing that the majority of the drivers in any given race do not have connects sufficient to give 0.111 latency, and put them in a pack, the end result is usually a massive crash with only the slightest of driver error. This starts a "pinball effect" of warping into other player's cars, with more collisions to figure (and the delays I've outlined above) and the next thing you know, it looks to every driver in the race like everyone else is warping badly! Actually, there are no fingers to point at anyone, other than one driver who might have made a small miscalculation in judgment.

In a nutshell, GPL can fairly accurately predict what the other driver's cars are doing remarkably well given the info passed in the data packets in normal racing, but if a collision occurs, the predictions seem to fail rather badly with the physics as modeled by the cars involved in the crash. This disparity between the predicted position of the car and the actual position reported back is the major cause of these massive pileups all the drivers blame on warping. It isn't actually warping as much as it is missed predictions and the catastrophic results of repositioning the cars once their true position post crash are known. Sure looks like warping, though! :)

- Doc Wynne

This is a very good analysis of crash dynamics, and the problems presented by attempting to predict car position and velocity in the wake of a collision in a high latency environment. It makes it quite clear why a crash at the start or T1 wreaks havoc. Now I understand better why it is worse in online racing than real life!

Thanks, Doc!

- Alison

Impossibly Short Lap Times and Other Evils

Concerning the above race, Doc also remarked:

Practice was very smooth, I never saw any cars warping, etc., although the one who was having problems connecting did manage a 5.14 sec. lap at Monza. :) I can see how this happens as well...he leaves the pits, makes enough of a lap to cross the line and start the lap timer, then gets discoed, then rejoins it immediately (and now he's like car #20 in line) and immediately pulls out of the pits, and by being near the end of the line, he has to cross the start/finish line to get on the track...he now trips the timer, and the car # and driver name are correct for GPL, so it must be a good lap...even if it is way out of range.

Actually, most impossibly short lap times are caused by clock smashing, not disconnects. When you host, you see clock smashes as giant warps: a car will suddenly disappear, and then reappear a long distance away. Sometimes the car's sound warbles strangely too. On the client where a clock smash occurs, this is accompanied by a startling flash of the screen and a brief interruption in sound.

If a clock smash occurs near the s/f line, it can cause the server to score a very short lap. For example, if the client car crosses the line, and then its clock gets smashed forward a half second, then it will be relocated behind the s/f line (from the host's perspective). It will then keep driving and cross the s/f line again a moment later.

It's also possible to gain an extra lap, lose a lap, get disqualified for going backwards, or get black flagged for cutting the course, all due to clock smashes which relocate the client's car some large distance.

I was having severe clock smashes due to a defective modem driver, and I've had a number of ultrafast laps during races, when obviously there was no chance that they could have been caused by disconnect/reconnect. Fortunately, I finally found the problem, and have been clock smash free for two days now. :-)

[For a more detailed discussion of the clock synchronization issue, including comments by Papyrus engineer Randy Cassidy, see the Bizarre Stuff! section in my GPL Online FAQ. Note that GPL 1.2 contains a far superior method of clock synchronization, which dramatically reduces the frequency of occurance of the problems described here. - ed.]

People who are having connection problems are also prone to have clock smashes, and vice versa. Clock smashes are large corrections in the client's internal clock, which are necessary if the client's clock gets out of sync with the host's clock. Unfortunately, if the timing data (constantly being exchanged between client and host) gets delayed, the client may assume that its clock is too far ahead, and smash it back. When data starts coming in a timely fashion again, the client will then smash its clock ahead again.

Several things can delay timing data: a saturated host CPU, which will thus be slow responding to the client's timing request; a saturated client CPU, which will thus be slow picking up the host's response to its timing data request; a spike in latency, which will delay the timing packets; clogged serial ports at either end, which will do the same thing.

So, people with slow computers who haven't cut graphics detail enough to get 36 fps, and people with bad internet connections (high and highly variable latency) are prone to both disconnects (which happen when too much data gets lost or delayed) and clock smashes (which happens when timing data gets lost or delayed).

[Some people can get away with running lower than 36 fps in online racing in GPL, while others can't. I have no idea why this is. The rule of thumb is, if you see more than occasional clock smashes, or you have problems staying connected to GPL races, get your frame rate up to 36 fps. - ed.]

Incidentally, the situation is aggravated when latency is high, because GPL times out on overdue data, and re-sends it, thus requiring more bandwidth. That's why sometimes one person with a bad connection or slow computer can cause a lot of people to disconnect. This will happen if the host's bandwidth is nearly saturated due to a lot of users. The re-transmission required by the unfortunate client saturates the bandwidth, causing delays, and people start dropping like flies.

I suspect Papy could write a patch to make the host compare the lap time against a set "minimum allowed" time for that track (like maybe 1:20 at Monza) and disallow anything under that time...that should keep host's lap records from getting hosed by this bug.

I've suggested that they do that. In the meantime, anyone who's had bogus lap records saved by GPL can simply delete the records.ini file in the appropriate track folder in GPL.

- Alison

Transfered to the wiki: 17.04.2009