It is, especially in hindsight, difficult to appreciate how big a deal DPMI was.
But to try to give you an idea: I was in an OS/2 seminar (because, well, OS/2 was the future at that point in time) when someone from Microsoft circulated photocopies of an early DPMI document.
This instantly divided the audience in two groups: those who got it (and abandoned the OS/2 hype train that instant), and those who did not.
When (some years later: Internet time was not yet invented either) Windows 95 came out with pretty much flawless DOS compatibility, whereas OS/2 was still struggling to run more than one DOS app simultaneously (with DOS apps still being very much the only revenue-positive game in town), OS/2 was effectively dead.
Sure, NT put the nails in the coffin, but it would be a few years before typical consumer/business hardware could reliably run that. With DPMI (an API, not even a product...), Microsoft settled the OS wars (as they were at the time...) decisively.
> “Windows 95 came out with pretty much flawless DOS compatibility, whereas OS/2 was still struggling to run more than one DOS app simultaneously”
This doesn’t match my recollection. Yes, 16-bit OS/2 1.x did poorly with DOS because it couldn’t use the 386 virtualization capabilities, but 32-bit OS/2 2.x was a great environment for multitasking DOS apps. It definitely supported DPMI too.
Many people ran complex apps like 3D Studio under OS/2 rather than plain DOS. It even had the ability to preemptively multitask Windows 3.0 apps which Windows itself didn’t.
IBM used to have a marketing slogan for OS/2:
“A better DOS than DOS and a better Windows than Windows.”
Unfortunately that was a fairly niche market after all. People didn’t want a somewhat better environment for running Windows apps in parallel sandboxes when it came with so much installation hassle and such high system requirements. With practically no native apps and losing the ability to run Windows soon (since IBM’s license only covered Windows 3.x), it had no chance.
When it came to games I had written I had relied on promises that Microsoft had made that dos virtualization would be quite seamless. My game used a pseudo retrace interrupt by carefully calibrating the timer every frame to generate an interrupt just before the screen retrace and adjusting the timer to compensate for drift.
This did not work in Windows95. OS/2 did it almost perfectly (it missed the occasional frame but not enough to matter)
I remember also that it was quite flaky. My recollection, probably wrong, was that DOS software that really pushed the system in certain ways wouldn't work right under Win95, especially audio stuff.
I used to work with Impulse Tracker and spend lots of time with the Demoscene and IIR there was a way to kill Win95 and drop back to pure DOS (or boot into DOS instead) and things worked fine there. Windows 95 in that sense still just felt like a very powerful shell over MS-DOS like Windows 3.1 and it wasn't until we got the NT derivatives (XP etc.) that I, and the demoscene, reluctantly gave up on DOS mode software.
It took the scene a very long time to bounce back from that transition, and in some ways it came out of it a very different scene.
tldr: https://devblogs.microsoft.com/oldnewthing/20160411-00/?p=93... Intel screwed up implementing v86 mode by missing one privileged accesses trap - Protected Mode program (CPL 3) executing CLI and expecting POPF to restore interrupt flag. CLI will correctly trap, but POPF is silently ignored AND doesnt trap. No easy workarounds available.
That's in protected mode. In V86 with IOPL < 3, both PUSHF and POPF will trap.
However I think some versions of Windows used to run V86 code with IOPL = 3. At least I remember being able to run "CLI, HLT" instructions in DEBUG inside a DOS box and locking up the entire system.
"Intel screwed up implementing Protected Mode by missing one privileged accesses trap - Protected Mode program (CPL 3, IOPL < 3) executing CLI and expecting POPF to restore interrupt flag. CLI will correctly trap, but POPF is silently ignored AND doesnt trap. No easy workarounds available."
One of the prominent victims is Doom. Works in Windows only due to hacks.
This story is mixing up the situations of 1987 with OS/2 1.x and of 1995 with DOS+Windows 95. By the time of the latter, OS/2 1.x was long past. 32-bit OS/2 used v8086 to run multiple Virtual DOS Machines and had DPMI support in them provided by the VDPMI.SYS virtual device driver.
> Why develop for OS/2 when you can use protected mode from DOS anyway?
GUI? Multitasking?
But was it even viable to use anything but DOS for gaming back then? WinG/DirectX took a good while to show up, and I'm not sure if OS/2 ever developed anything comparable.
Windows gaming was mostly things like chess and solitaire until then.
Unless you actually bought an IBM machine, it probably didn't have OS/2 installed. So you would need to actually buy OS/2, (and also a RAM upgrade, with OS/2 needing more than Windows 3.x), and if you wanted to run (16bit) Windows Apps, I believe you needed to buy the more expensive "Blue Spine" version.
I knew of a few people who got OS/2 as a pack-in with a new PC clone from a big store chain in that time (it was Escom I believe). Apparently, IBM also tried bundling as a way to increase market share.
> whereas OS/2 was still struggling to run more than one DOS app simultaneously
That was hardly universal. I remember running multiple dos-native _games_ on os/2. Yeah, they were slow; they were slow when running under DOS as the only app running, but they ran. I never had any compatibility issues. It was one of the most "fun" OS's I've ever hit.
> Miraculously, most games just worked despite running under a different DPMI server from what they were originally developed with. There were occasional issues with specific games. Popular ones include games which assumed that all memory was physical and games which assumed the interrupt flag was unvirtualized, but for the most part, things worked well enough that the remaining issues could be treated as app-specific bugs.
It's funny because at that moment in time I was running DOS games, and most of them worked without issues and I was thinking that Win95, wasn't too different and just a UI, like it was Win 3.1x
I felt like a wizard, running multiple windowed DOS VGA games, along with Word and a video (Weezer!) playing all at the same time on Windows 95
Something about the multitasking in Win95 (among other things) was so different in Windows 3.1, even on the same hardware. You could clearly see processes passing control to each other and updates were so slow.
> Something about the multitasking in Win95 (among other things) was so different in Windows 3.1, even on the same hardware. You could clearly see processes passing control to each other and updates were so slow.
That's exactly what was different: Windows 3.1 used only co-operative multitasking (for 16-bit Windows apps anyway), where apps have to yield control to eachother, whereas Windows 95 added pre-emptive multitasking where apps will give up control when the OS wants them to, whether they like it or not.
> DOS/32 Advanced DOS Extender provides full compatibility with Watcom C/C++ and features one of the most complete software emulations of DOS/4GW, thus allowing the users to "plug" it into the protected mode programs which use DOS/4G, DOS/4GW, DOS/4GW Professional and any other compatible DOS Extenders without the need to modify a single line of code.
>
> DOS/32 Advanced DOS Extender has been designed to be fast, flexible and reliable. It features very fast mode translation and interrupt servicing, ability to configure and bind the DOS Extender to applications, protected mode executable compression, support for allocation of up to 2 GB of memory, Null-pointer protection and more.
DOS/4GW was a limited version of DOS/4G. Early versions had trouble with secondary DMA channels on the ISA bus (wreaking havoc with certain 16 bit soundcards). The 4GW version especially was also considered slow and bloated, and allows only up to 64MB of memory to be used. DOS/32A allowed allocation of up to 2GB, and has better performance and a smaller binary.
A side question, any recommendation on emulator and books if I want to replicate what Carmack did for Commander Keen? The keen dream source code uses C and assembly so I'll do the same.
Check out 86Box[0], unlike Dosbox it does full system emulation and tries to be cycle accurate (or cycle accurate enough) so that you will notice little to no discrepancies between the emulated hardware and the real one.
86Box was forked from PCem which is kinda similar but PCem development has practically stopped and all development nowadays goes towards 86Box.
Also FWIW do not use plain Dosbox, it has longstanding bugs that haven't been fixed in years (some have been fixed in SVN but they haven't made a new major release for a very long time). Use Dosbox-X or at least Dosbox Staging instead (Dosbos-X is more accurate). As an example Post Apocalyptic Petra[1], a game i made ~3 years ago for a DOS game jam doesn't work on plain Dosbox because of some timing issues (you may get a black screen or visual glitches) - you need either Dosbox-X or Dosbox Staging.
Also note that these projects have slightly different goals: 86Box tries to emulate the hardware as precisely as it can whereas Dosbox(-X/Staging/etc) tries to provide a compatibility layer for running older software (plain Dosbox is only focused on games but other forks expand that to include all software). This means that Dosbox(/etc) will be more likely to ignore "bad" programming that wouldn't work on (or only work on some) real hardware - e.g. its Sound Blaster emulation is very forgiving and if you write some SB code and test it only on Dosbox chances are it wont work on the real hardware.
C isn't C++. Most of what you know today applies to older compilers as well, with only a handful of features added over the years. You aren't going all the way back to K&R calling syntax here, I'm assuming you will target C89. You will just have to remember to declare all of your variables at the top of each function and the stdint types won't be available. There are a few other things added over the years, but its going to be pretty much the C you know and love.
> You will just have to remember to declare all of your variables at the top of each function
This is still idiomatic to the C community, even if it isn't strictly required any longer. About the only place it's ignored is inline variables for loops.
I write C primarily for hobby projects, and I only started with C ~10 years ago, so I'm likely in the minority that considers this a code smell for new C code. I don't want to needlessly extend the scope of my local variables!
Different story of course if you're targeting C89 or older, which I'm actually doing in my current project. I'm writing software on and for my old 68k Macintosh.
We might be saying the same thing here, but modern C scopes variables down to the block. So if you have a variable declared for instance inside of a for loop or if statement, that variable is out of scope once you exit the block.
This will fail to compile with an "undeclared identifier" error:
Pedantically you are correct, but it is and was common to use C++ compilers as a slightly more convenient dialect of C. This gives you the performance and simplicity of the C language with some very nice quality of life improvements.
With few exceptions, C++ can mostly be treated as a superset of C.
What I was trying to say is that if you're comfortable using modern C++ and try to switch back to a C++ compiler from the early 90s you are going to find a lot of the features you regularly use suddenly being absent. With C this is much less of an issue.
IIRC they were also available as “non-standard” feature in compilers at the time before being added to c99 (e.g. I’m pretty sure gcc in the late 90s supported all of these if you didn’t force c89 compatibility).
I know you said you're using openwatcom, but if you're not familiar with DOS, I'd suggest running through https://fabiensanglard.net/Compile_Like_Its_1992/index.php to build Wolfenstein with Borland C 3.1. The public Keen Dreams repo is very similarly structured and will build with BC3.1 as well.
Getting data into and out of the more accurate emulators can be a pain. Personally I use VirtualBox with a 32-bit Windows image to do builds. It has network access and VirtualBox shared folders connect it to my desktop. It can run DOS and 32-bit windows toolchains, which covers everything that existed in the DOS programming days. To get things to either PCem, or my real 386sx, I mount a virtual floppy image in VirtualBox, or connect a Sabrent USB drive to write a real floppy, respectively.
(Something like PC-NFS or Microsoft's DOS SMB client would be the easiest way to get data into PCem/86box/etc. I'm yet to set it up).
I would check out the Andre Lamothe DOS game programming books
You will also want to start with a 16-bit C compiler like Borland Turbo C or Microsoft C, a lot of the code is very different from the 32 but protected mode versions
There's always DJGPP which Quake was written in, but again 16-bit DOS code isn't going to run as-is.
Thanks. I used to do some C++/sdl2 game programming just for fun but I think this project needs more study.
TBH I don't know if Commander Keen is using protected mode. I do not understand the code and I assume not. I need to know how to program and compile for DOS 16-bit, as you said, and how to inline assembly, and how to use assembly to put pixels on screen using a virtual ega card provided by dosbox.
pcem/86box emulate an entire system (cpu, memory, bus, drives, ports, etc) while dosbox simulates a system below DOS. It may seem pedantic, but a simulation in essence "displays the right answer" while an emulator "shows your work". Especially with old games where devs were trying to squeeze performance out of systems, there are many timing or undocumented behavior tricks they used that a simulator may not replicate properly.
Snap, my incomprehensible article made you want to program the whole thing? Sorry about that!
As a side note, if you want to go Commander Keen style, you won't have the pleasure to use DOS/4G. CK, alike Wolfenstein 3D, use real mode. Personnaly, I find having to deal with near/far pointers a huge turn off from programming this way.
Hi Fabien thanks a lot for the books and articles! TBF I only understand a small part of them but I find them fascinating.
I'll try out real mode and see what happens. I read about near/far pointers some 30+ years ago in one of my father's programming books so that's the only thing I know about them -- I heard about them :D
I think this one is already a mirror, with the original lost in time, but not sure... I had the .txt files as a collection downloaded in 2004 or so, the readme file in it says the previous location was on an Oulu University FTP site.
>Miraculously, most games just worked despite running under a different DPMI server from what they were originally developed with. There were occasional issues with specific games. Popular ones include games which assumed that all memory was physical and games which assumed the interrupt flag was unvirtualized, but for the most part, things worked well enough that the remaining issues could be treated as app-specific bugs.
That is Microsoft-speak for "We're going to patch SimCity because we need that to run absolutely." and I have nothing but the utmost respect and admiration.
Yes this particular story drove into me the importance of backward compatibility, and how much Microsoft actually cared about lots of things at a time they were getting a lot of flak for their OSes.
It also made me realize that changing anything in windows is probably very expensive.
Fun fact: the DPMI server was active on Windows 3.x even when DOS programs weren't running. This was taken advantage by Watcom compilers for their "win386" target to generate 32bit code that ran under Windows 3.x (this was unrelated to Win32s and didn't require it).
Reminds me of lots of fun memory in last but great days of DOS as mainstream and dawn of Win32.
DOS/4GW was not the only game in town, there were a few of those 32bit extenders, I recall using sth called WDosX to run Delphi compiled (win32) exe in DOS. DOS/4GW was probably the most popular one partly thanks to the popularity of Watcom C compiler.
Pharlap was doing Pmode ;
Zortech C (bought later by Symantec), were doing X32 which was compatible with Pharlap but like 5 times smaller ; djgpp (gnu cpp for dos/win then) had a variation
And you could mix and match (if u have written good parametrized makefile/s that is), i had like 6 compilers vs 3-4 diff.extenders (but not all combinations possible)
interesting times these were. internet? what's that?
I wish Apple could figure out something like Proton for MacOS. It feels like a relatively small investment in this area could unlock a huge catalogue of classic games. It's the only thing I feel I'm missing by switching from Linux.
This is true. Worthy of note: this toolkit was released fairly recently and there hasn’t been much time yet for commercial game devs to consider using it. Here’s hoping they do.
It's against the terms to use the kit directly - the kit is intended to be a porting aid, not an emulation/compatibility layer. Which unfortunately ensured it won't be used much until Apple sees the light and changes the terms.
What's stopping you now? Apple's devkit for game porting already allows you to run many Windows games without modifications. They've essentially taken Proton and implemented their own proprietary Vulkan backend.
I wouldn't expect too much from Apple in terms of game compatibility (they'd much rather see developers port games and sell them in the App Store). Traditionally, they have put very little care into video games on their desktop grade machines.
You could probably write a lutris-like Game Porting Toolkit wrapper, I imagine you may be able to make some cash off of that too.
> Traditionally, they have put very little care into video games on their desktop grade machines.
MacOS got lots of OpenGL fixes around 2010 when Steam for Mac launched. Problem was most Macs at the time didn't have competitive graphics hardware so unless you had a Mac Pro only older games ran properly.
The M series GPUs still aren't very powerful gaming chips, especially for their price range and the very high resolutions macOS is built for.
The Mac wasn't really built for gaming and there's nothing wrong with that. Sadly, that does make me believe that Apple won't put serious effort into cross platform gaming like Valve has done on Linux; they will probably bring out porting tools every now and then, but I don't think they'll make game compatibility a first class citizen like Valve has done for Linux. If that was their intention, they wouldn't have put the distribution restrictions on their game porting toolkit.
I'm not sure how a Vulkan-to-Metal translation engine is supposed to help with porting, though; including a custom Wine build makes sense, but without Apple's Vulkan/DX translation layers you won't get much out of Wine directly. It's also not as if the toolkit spits out code that'll turn Vulkan calls into Metal calls. The whole thing just feels like a proof of concept to me, a demonstration in the sense of "look at the 30-60fps you can get on low-medium settings on our hardware!" rather than an intent to actually port existing games to macOS.
Ha. I barely recall details (it was 30 years ago) but I remember there was a trick to somehow run Windows 3.x to make the game believe your system had more RAM that it actually did. Don't remember details, but remember it _mostly worked_ (swapping was an issue and some games would not work). This - https://devblogs.microsoft.com/oldnewthing/20160328-00/?p=93... - explains it.
I believe I had to do that to run some games in SVGA mode. It might've been KKND or some other game...
I've used this once! Warcraft 2, playing against a friend over a null-modem, it all started out great but the swapping became worse and worse during the game. It lagged both our games, but he (actually having the physical RAM required) at least had a responsive UI, that was too big of competitive advantage.
I don't remember which game it was but it crashed with OOM under DOS[0] but run fine under Win3.1 and AFAIR didn't even bothered to swap, or the swap times were minuscule.
[0] It didn't help what I had a 40Mb HDD as the primary master, because nobody knew the proper jumper settings to change it to the slave and 120Mb DBLSPACEed WD as the slave. I think the regular working setup had only ~580Kb of low memory.
If I recall correctly, EMM386 would actually switch to protected mode an run DOS as a v86 application. It provided VCPI and DPMI apis. Their use case was a bit different (VCPI would allow you to run your code in ring0, while DPMI would run on ring2.
Fun fact, Win95/98 could of course run 32 bit DPMI apps without a problem, and many of the relevant OS interrupts such as 21h could be invoked directly from protected mode.
Interesting reading this, just at a point when dosemu2 got a load of fixes for DPMI 1 (note -DPMI programs have worked for a long time in dosemu / dosemu 2).
The article is written with the assumption that various intricate historical technical facts are known to the reader and others aren't (hence they are explained). But still it fit my knowledge profile perfectly as these assumptions were exactly true for me.
But to try to give you an idea: I was in an OS/2 seminar (because, well, OS/2 was the future at that point in time) when someone from Microsoft circulated photocopies of an early DPMI document.
This instantly divided the audience in two groups: those who got it (and abandoned the OS/2 hype train that instant), and those who did not.
When (some years later: Internet time was not yet invented either) Windows 95 came out with pretty much flawless DOS compatibility, whereas OS/2 was still struggling to run more than one DOS app simultaneously (with DOS apps still being very much the only revenue-positive game in town), OS/2 was effectively dead.
Sure, NT put the nails in the coffin, but it would be a few years before typical consumer/business hardware could reliably run that. With DPMI (an API, not even a product...), Microsoft settled the OS wars (as they were at the time...) decisively.