The recording FAQ

Uwe Girlich

v1.1.7, 10/24/2009

These are the frequently asked questions with answers concerning game recordings.


Table of Contents
1. General
1.1. About the FAQ
1.1.1. Why just another FAQ?
1.1.2. What do I find in this FAQ?
1.2. Personal questions
1.2.1. Why all your stuff isn't Windows?? friendly?
1.2.2. What's your email address?
1.3. Game types
1.3.1. What games are covered in this FAQ?
2. Playback problems
2.1. Non-working recording files
2.1.1. How can I replay a multi-level recording?
2.1.1.1. Quake and Hexen II
2.1.1.2. Quake II
2.2. Too long recordings
2.2.1. How can I split my large recording into single-level pieces?
3. Format conversions
3.1. Recording formats
3.1.1. Is there a QWD < - > DEM converer?
3.2. Movie formats
3.2.1. How can I convert a Quake DEM file to a MPEG file?
3.2.2. How can I convert a Quake II DM2 file to a MPEG file?
4. Tools
4.1. LMPC
4.1.1. What stands LMPC for?
4.1.2. Does LMPC accepts long filenames ?
4.1.3. What are the different versions of LMPC ?
4.1.4. Will (insert your favorite OS here) ever be supported?
4.1.5. How do the both Win32 versions differ?
4.1.6. Are there GUI versions of your command line tools?
4.1.7. What about the QWD support in LMPC?
4.1.8. What about the Hexen II support in LMPC?
4.1.9. What about the Quake II support in LMPC?
4.1.10. What about Quake III Arena support in LMPC?
4.1.11. What about the (insert your favorite game here) support in LMPC?
4.1.12. I found an error in LMPC, what shall I do?
5. Additional help sources
5.1. Network protocols
5.1.1. What is the difference between the QWD file format and the QuakeWorld network protocol?
5.1.2. What is the difference between the DM2 file format and the Quake II network protocol?
5.1.3. What is the difference between the DM3 file format and the Quake III Arena network protocol?
6. Version History

1. General

1.1. About the FAQ

1.1.1. Why just another FAQ?

Because I get so many mails asking me the same things over and over again.


1.1.2. What do I find in this FAQ?

Information on game recordings, N O T covered by my standard documents:

The Unofficial LMP format description

http://demospecs.half-empty.de/lmp

The Unofficial DMO format description

http://demospecs.half-empty.de/dmo

The Unofficial DEM format description

http://demospecs.half-empty.de/dem

The Unofficial QWD format description

http://demospecs.half-empty.de/qwd

The Unofficial DM2 format description

http://demospecs.half-empty.de/dm2

LMPC user's guide

http://demospecs.half-empty.de/lmpc-alpha, http://demospecs.half-empty.de/lmpc-beta or http://demospecs.half-empty.de/lmpc-stable


1.2. Personal questions

1.2.1. Why all your stuff isn't Windows?? friendly?

I have no Windows?? at home and also not at work (I'm working for a UNIX software company). I write all stuff with Linux and have to cope with it.


1.2.2. What's your email address?

This is a difficult question. I have currently two main addresses but will most likely answer from my work address.

Uwe.Girlich@philosys.de

is my mailbox at work. It is currently my main mailbox and I will answer mails to other boxes from here.

uwe@half-empty.de

is my mailbox at Half-Empty. I get the mail from there about once a day and will certainly answer it from my mailbox at work.


1.3. Game types

1.3.1. What games are covered in this FAQ?

Quake

 

QuakeWorld

 

Hexen II

 

Quake II

 

Quake III Arena

 


2. Playback problems

2.1. Non-working recording files

2.1.1. How can I replay a multi-level recording?

2.1.1.1. Quake and Hexen II

Quake and Hexen II have problems with the play-back of multi-level recordings. The reason are blocks like

block {
 camera 0.00000000 0.00000000 0.00000000;
 stufftext "reconnect\n";
}
between the different levels. The reconnect command confuses the play-back totally.

The stupid solution is to use the timedemo console command.

LMPC can repair multi-level DEM files with the --fix-multi command line option. It can be used on the fly (as an additional parameter to any kind of conversion) or with a single binary DEM file as "in-place editing".

During the level change of such a patched DEM file there might be a never ending sound effect but it will stop after the next level start.


2.1.1.2. Quake II

Quake II has similar, if not more severe problems with multi-level recordings. Because of the mission character of the game (multiple level changes to reach a single goal) it is vitally important to be able to replay multi-level DM2 files.

It is possible to chain single-level DM2 files, if only the last one keeps its endblock;. This leads directly to the solution for multi-level DM2 files:

  • betweenblock; has to disappear.

  • reconnect; has to disappear.

  • stufftext "reconnect\n"; has to disappear.

  • isdemo 0; in serverdata messages should be isdemo 1;.

  • stufftext "cmd configstrings key index\n"; has to disappear.

  • stufftext "cmd baselines key index\n"; has to disappear.

  • stufftext "precache key\n"; has to be changed to stufftext "precache\n".

  • stufftext "record filename\n"; has to disappear.

  • download { … } has to disappear.

All these things can be done with the Perl script MULDER, the MUlti-Level DEmo Repair kit at http://demospecs.half-empty.de/misc/mulder.html. It acts on DM2 text files only.

LMPC can repair multi-level DM2 files with the --fix-multi command line option. It can be used on the fly (as an additional parameter to any kind of conversion) or with a single binary DM2 file as "in-place editing".


2.2. Too long recordings

2.2.1. How can I split my large recording into single-level pieces?

First, you have to repair the demo file according to the section Section 2.1.1. Then you can use LMPC to convert it into a text file. Now use any decent text editor or simpler a script (in your favourite script language) to split the recording and insert the necessary beginnings (for DEM files: CD track) or endings (for DM2 files: the endblock). Convert these many text files back to binary with LMPC.

I have no experience with multi-level QWD files.

You can do the repair and split part for DM2 files with my Perl script MULDER, the MUlti-Level DEmo Repair kit at http://demospecs.half-empty.de/misc/mulder.html. Just use a printf format place-holder (like %d) in the output name to create one file per level:

mulder infile.dm2 outfile-%02d.dm2
creates output files with the names outfile-01.dm2, outfile-02.dm2 etc.

Future versions of LMPC may be able to do the same.


3. Format conversions

3.1. Recording formats

3.1.1. Is there a QWD < - > DEM converer?

I'll never write such a program but F@11 has a ``load QWD & save DEM'' feature. Read the source code of Quake and QuakeWorld and write your own.


3.2. Movie formats

3.2.1. How can I convert a Quake DEM file to a MPEG file?

Very easily. Quake can write up to 100 screenshots (PCX files) of the current screen. So it's simply a question of collecting all screenshots and MPEG encoding.

Some simple steps to reach this goal:

  • Quake writes 4 frames after the screenshot a message (Wrote quake??.pcx) to the Quake screen (top line). A 1 byte patch of the executable can remove the message.

    There are so many Quake versions out there so I think it should be easier to describe what to do: Find in the binary the message string `Wrote %s' This is the printf(3) format string. You can now change it to `        ' (this changes the message to some spaces) or (but this may not work) `\000rote %s' (the byte 0x00, not \, 0, 0, 0) (this collapses the printf(3) - message to nothing).

  • Take the DEM file, split it in 100 frames parts. Watch out for colour and number changes, so you may do it with DEMcut (which is included in the LMPC package) and its m command.

  • Convert it to text with LMPC.

  • Insert a

    stufftext "screenshot\n";
    command in every frame of interest.

  • Convert it back to binary with LMPC.

  • Use Quake to perform a playback of one DEM file after another.

  • Use any picture manipulation program like

    to improve the quality (gamma correction, aspect ratio, format conversion etc.).

  • Use any MPEG encoder like

    to encode the MPEG file and remove the encoded screen-shot files. Take care to have at least as much pictures as the GOP length in the MPEG file available. The MPEG encoder reads the files in a different order than usual. It reads first the picture what will become an I frame, than the P frame and the B frames inbetween are the final ones.

  • Note: the PCX file name algorithm is very simple: Quake tries the first name (quake00.pcx, than the second (quake01.pcx) and so on. The first unused PCX name will be used. So it is better to move the PCX files out of the way after the playback of the first chunk.

  • A really good variant would first find the split points in the file, and then delegate specific tasks to many (over the network) connected computers. Even the MPEG encoding can be parallelised.

The just presented solution should be regarded as way to write a script to do the different tasks. Don't even try to do all these things ``by hand''.


3.2.2. How can I convert a Quake II DM2 file to a MPEG file?

This is a bit more difficult, because Quake II uses a fix time gap of 100ms (10Hz) between the frames in the DM2 file. So you end up (if you adopt the Quake method) with a totally unacceptable 10Hz movie. Quake II uses a sophisticated prediction scheme to interpolate between these 10 frames per second but you won't get (with stufftext "screenshot\n";) any of these interpolated frames.

So the only good method is to grab the actual picture from the screen. This has to be done much faster than these mentioned 10Hz to get real movie speed (25 or 30 frames per second).

The rest can to be done as usual (picture manipulating, MPEG encoding etc.).

As far as I know there is no continual working screen grabber but since all modern operating systems use some kind of run-time-linking it should be very easy to overload the actual drawing routine (in X11: XShmPutImage) with a different one, which stores the data in a file. Since the file writing goes much slower than a screen update, something has to be done with the speed of the game. To do this, you have to find the time synchronisation routine (gettimeofday or select) and overload this too.

The really stupid (but often found) method is to use a graphics card card with a video-out feature and feed the video signal back into a grabber card.


4. Tools

4.1. LMPC

4.1.1. What stands LMPC for?

It used to stand for ``LMP Control Centre'' and I inserted later the DMO, DEM and QWD format but the final ``LMP/DMO/DEM/QWD Control Centre'' was such a stupid name and I decided to rename it to ``Little Movie Processing Centre''.


4.1.2. Does LMPC accepts long filenames ?

Yes of course. I didn't include any file name restriction but the underlying OS may have problems with it. To prevent file name problems the MS-DOS binary (lmpc.exe) even changes all file names to lower case.


4.1.3. What are the different versions of LMPC ?

There are 6 versions of the current LMPC 3.4.4 available:

lmpc-3.4.4.tar.gz

(download from SourceForge) is ``the Source Package''. It comes with no binaries but contains the full source code with an Autoconf/Automake configure file for easy compilation on any (Unix) system. The documentation source written in DocBook SGML is included as well.

lmpc-3.4.4-f.tar.gz

(download from SourceForge) is ``the Full Package''. Everything from all other packages and a PDF documentation is included.

lmpc344d.zip

(download from SourceForge) is ``the MS-DOS Binary Package''. It comes with lmpc.exe, demcut.exe, dema and a HTML documentation.

lmpc-3.4.4-l.tar.gz

(download from SourceForge) is ``the Linux Binary Package''. It comes with lmpc, demcut, dbs, dema and a HTML documentation. The Linux binaries are compiled for Intel PCs.

lmpc344w.zip

(download from SourceForge) is the ``Win32 (Cygwin) Binary Package''. It comes with lmpc32.exe, demcut32.exe, dbs32.exe, dema and a HTML documentation.

lmpc344m.zip

(download from SourceForge) is the ``Win32 (MinGW) Binary Package''. It comes with lmpcm.exe, demcutm.exe, dema and a HTML documentation.


4.1.4. Will (insert your favorite OS here) ever be supported?

I wrote LMPC in simple ANSI C with some parts in Lex and Yacc, so you should be able to port it to any decent OS. I have access to some UNIX (not Linux) machines. If you need a binary for such systems please ask. I create some source code files during the compilation process with Perl scripts, so you definitely need Perl on your computer.

The main distribution packages were build at home with cross-compilers and I think that I wasted yet enough hard-disk space with three of them.


4.1.5. How do the both Win32 versions differ?

The Cygwin version needs the special run-time DLL. Get it from http://demospecs.half-empty.de/misc/win32.html.

The MinGW version needs nothing else but is reported to perform a bit slower.


4.1.6. Are there GUI versions of your command line tools?

GUI rules the world but it is very difficult to maintain a GUI program for different OS's. I think about wxPerl or Qt http://www.troll.no for such a task. You are welcome to write such a GUI program.


4.1.7. What about the QWD support in LMPC?

QuakeWorld QWD files are fully supported from version 3.1.9 on.


4.1.8. What about the Hexen II support in LMPC?

I abandoned the Hexen II support in LMPC and the DEM specs. It's time to move on. Actually nobody ever aked for a full working Hexen II LMPC.


4.1.9. What about the Quake II support in LMPC?

Quake II DM2 files are fully supported from version 3.1.7 on.


4.1.10. What about Quake III Arena support in LMPC?

Quake III Arena DM3 (dm_68) files are fully supported from version 3.4.1 on.

Due to legal problems I could not add this earlier but after the source code of Quake III Arena was published under the GPL, I added support for it in LMPC.


4.1.11. What about the (insert your favorite game here) support in LMPC?

I may support any Quake 1 and Quake 2 derived programs like Hexen 2, Sin, Heretic II, Half-Life, Kingpin but all this takes time, really much time.

I think I'll concentrate on the original id games. I'm currently in a total rewrite of LMPC, which makes it much more modular, so maybe someone other is able to provide "plug-in" modules for some games.

The idea is to write all the knowledge to parse a particular demo format in a single XML file. I hope to create the documentation and the source code out of the same XML file.

If your favorite game does not exist in a Linux version LMPC won't support it.


4.1.12. I found an error in LMPC, what shall I do?

The best way is to get the source code, fix the problem and mail me the patch file.

If you are not able to do this (for whatever reason), please send a full bug report with all of the following points:

  • Exact version of LMPC are you using (output of lmpc -v).

  • Name and version of the OS you are using.

  • Information how you compiled LMPC or which precompiled binary you are using.

  • Recording type (LMP,DMO,DEM,QWD,DM2,DM3)! This is very important!

  • Full error message of LMPC.

  • Return code of LMPC.

  • If you can, please run it again under a debugger and get the stack trace at the crash point.

  • Exact description of what happended and what you expected to happen.

  • If possible, send the shortest recording which can cause the error by mail or place it on a FTP or HTTP server.

If you do it this way, you can expect a very fast response with the error fixed. If you send only some senseless things like ``I couldn't recompile my demo after decompiling it'' I won't pay much attention to you at all.


5. Additional help sources

5.1. Network protocols

5.1.1. What is the difference between the QWD file format and the QuakeWorld network protocol?

Read the source code of QuakeWorld.

There are many differences. Read the QuakeWorld Network Protocol Specs at http://www.ens.fr/~cridlig/bot/qwspec21.html for the network part and my Unofficial QWD format description at http://demospecs.half-empty.de/qwd for the file format part. It seems that the QuakeWorld Network Protocol Specs are unavailable at the moment (january 1999) but I hope to include them in my own QWD specs soon.


5.1.2. What is the difference between the DM2 file format and the Quake II network protocol?

Read the source code of Quake II.

There are many differences. Read the Quake II network specs area at the Pandora bot site www.opt-sci.Arizona.EDU/Pandora/q2 for the network part and my Unofficial DM2 description at http://demospecs.half-empty.de/dm2 for the file format part.


5.1.3. What is the difference between the DM3 file format and the Quake III Arena network protocol?

Read the source code of Quake III Arena.

Quake III Arena uses a (simple) encryption scheme to secure the network protocol. This encryption is not included in DM3 files.


6. Version History

0.0.1, 1st September, 1997

  • First version completed.

0.0.2, 2nd September, 1997

  • Hints on QWD, Hexen II.

  • More links for LMPC.

0.0.3, 17 September, 1997

  • Multi-level playback problem.

0.0.4, 11 October, 1997

  • QuakeWorld network specs mentioned.

0.0.5, 23 October, 1997

  • Quake II DM2 files mentioned.

0.0.6, 29 December, 1997

  • Quake II DM2 support in LMPC mentioned.

0.0.7, 2nd January, 1998

  • New versions for DM2 specs and LMPC included.

0.0.8, 15 March, 1998

  • New versions for LMPC included.

  • PlanetQuake is the new home.

  • SGML-Tools 1.0.5 used.

0.0.9, 17 June, 1998

  • New versions for LMPC included.

  • Some more Quake II and HexenWorld hints.

1.0.0, 15 July, 1998

  • SGML-Tools 1.0.7 used.

  • New versions for LMPC included.

1.0.1, 16 August, 1998

  • New versions for LMPC included.

  • QuakeWorld support in LMPC mentioned.

  • Dead links to FTP mirrors corrected.

  • New bug report section.

1.0.2, 20 August, 1998

  • MUlti-Level DEmo Repair kit mentioned.

1.0.3, 25 August, 1998

  • Multi-level problem corrected.

1.0.4, 6 September, 1998

  • New stable release for LMPC included.

  • Multi-level section restructured.

  • DM2 network protocol mentioned.

1.0.5, 8 September, 1998

  • New stable release for LMPC included.

1.0.6, 8 January, 1999

  • New stable release for LMPC included.

1.0.7, 12 January, 1999

  • New stable release for LMPC included.

  • Hint on the vanishing QuakeWorld network protocol specs.

1.0.8, 7 April, 1999

  • More hints on movie creating.

  • Some hints on file formats for the distribution packages.

  • New stable release for LMPC included.

  • Section on multi-level DM2 files corrected.

1.0.9, 29 April, 1999

  • New stable release for LMPC included.

  • Some hints on the DM3 problems.

  • SGML-Tools 1.0.9 used.

1.1.0, 9 May, 1999

  • New stable release for LMPC included.

1.1.1, 3 June, 1999

  • New stable release for LMPC included.

1.1.2, 18 August, 1999

  • New stable release for LMPC included.

  • Multi-level repair section enlarged.

1.1.3, 23 January, 2000

  • New stable release for LMPC included.

  • More hints on future game support.

1.1.4, 4 March, 2006

  • Hints on Quake III Arena completely changed.

  • Too simple and too old questions removed.

  • Links updated.

1.1.5, 26 August, 2006

  • Typos corrected.

  • Permanent link to dm2 repair created.

  • Download links updated.

1.1.6, 18 February, 2009

  • Link to ImageMagick corrected. Thanks to Amy Hazelton.

1.1.7, 23 October, 2009

  • Half-Empty is the new home. PlanetQuake will be removed soon or is already when you read this.