The Impossible

Uncategorized

EDIT: The update has been available since last Friday. You can check it out by downloading the latest update from FreeSO servers, then following the instructions later in this post!

Three years ago, any chance of anyone reviving The Sims Online looked very slim. Any existing project seemed to either fall off in activity, be a hoax in the first place, or burn hopeful fans with controversial donations for a server that was never meant to be. Despite all the ruckus, three years ago is when we got the first glimmer of hope that TSO would be back again – a convincing UI recreation of the initial few screens of the game including Create A Sim, with a lot of backend work to load important game content. Developed by Afr0 and ddfczm, this recreation was called Project Dollhouse, and it’s what inspired me to work towards this same goal. Despite exciting developments in SimAntics emulation by me and ddfczm, with ingame footage of TSO objects somewhat working, this community stayed at only a few members – nobody believed anymore in TSO ever returning. Our community slowly grew as I implemented the online sandbox mode and split into FreeSO, and people finally started believing that it was possible. The launch into beta was a spectacular surprise.

Today, we’ve almost done it. It has been a heck of a long journey, and finally people don’t have to believe in something anymore – it’s right here. A fully re-implemented TSO client and server is available for all to use, with source available so it can be developed further by anyone in the future, maybe tens of years from now. The ability to play TSO will never truly disappear, thanks to the efforts of its community.

Recently, I attempted to port The Sims 1 to mobile by myself, something many users never would have believed would work until recently. Unfortunately this was not meant to be, but I demonstrated that it was never impossible, and that it can even be done well by an independent developer.

To keep up with my trend of doing stupid things nobody believes in, I decided to make good on a crazy idea I shared three years ago, and have even mocked on one april fools since it felt so unfeasible at the time. I had an idea that The Sims and The Sims Online could potentially be rendered and playable in 3D, in real time, using mesh reconstruction techniques. Our next update will have an experimental mode showing that this is indeed possible!

I’m guessing your immediate thoughts from seeing our header image is, “how is this even remotely possible? the objects in this game are 2D sprites”. This article will also explain how it wasn’t as impossible as it seemed, and explain how it was achieved.

Update timing and changes

The update will release within the week. It will include the 3D mode shown in the video (see later in this post for more info), but also several critical bug fixes including fixes for interactions, disappearing purchased objects, lot save mishaps and many common SimAntics exceptions.

Note that in 3D mode, the first time you see each drawgroup the game will freeze or stutter heavily. This is because it is generating the 3D meshes the game uses from the sprites – these are not included with the FreeSO client!

The update also includes two enhancements for 2D mode. You can now use the mouse wheel to zoom in and out, and our new 3D wall component can be used in 2D mode to generate 3D wall shadows. You’ll find a toggle for this in the options menu.

3D in Simitone for Windows

Though Simitone can no longer be developed for mobile devices, the source is still available and buildable on windows desktop. Since Simitone runs on the FreeSO engine, these changes work in it as well. Gameplay is still in the same state as seen in the last video, so it’s not entirely playable.

History

The idea of converting TS1/TSO objects to 3D was actually something I was interested in even before I started working on Project Dollhouse or FreeSO. While Niotso and PD were still active, I frequented the Niotso IRC on hackint, where we discussed the various formats in the game.

Most other isometric games simply draw sprites from back to front, which works well for most games, assuming that the sprites are never meant to intersect each other. The Sims is a rather special case, since object sprites intersecting with avatars is a key requirement for most of the objects – specific parts of a shower, for example, must appear over the sim while they are in it. For this purpose, all object sprites in The Sims have a little extra depth… in that they literally store a depth value for each pixel in the sprite, as it was when it was originally rendered from a 3D model. This is called its z-buffer, and allows per pixel depth comparison between objects and avatars.

I created a javascript tool for experimenting with TS1 sprite rendering, which mostly served as a way to tweak and examine how the z-buffers stored in object sprites should be used to perform depth ordering between objects and avatars that might intersect.

An object rendered alongside its Z and alpha channels. The Z channel allows pixel perfect intersection with other objects and avatars.

http://www.niotso.org/zbuffer/

This was not the only function of this renderer, though. A custom version I made was actually able to render objects from non-standard angles, using the z buffers and some vector math to offset their pixels to a new position in 3D space. (press 1, 2 to rotate)

http://freeso.org/zbuffer/index.html

This was a good proof of concept, but obviously this kind of 2D per-pixel transformation is too slow to run in real-time on a lot. It was clear a true solution would need to generate a 3D model which could be transformed and rendered on the GPU.

You could say that this has always been the secondary goal of this project, though obviously I never like to get people’s expectations up if they might be disappointed. It’s always been on my mind, either way.

Development

Developing the whole thing into a playable state took about a week. Not kidding – since the entire game was already there, and terrain was already in 3D, it didn’t take long to get a prototype working. The first steps were simply generating meshes for existing objects in the game and displaying them by themselves in volcanic. This was done to test how viable applying such a technique to the whole game might be, on a per object basis.

 

These triangle meshes were literally generated as heightmaps of the z-buffers, without any tricks. It became clear that these meshes would have too many triangles to render 1000-fold on a typical TSO lot, so the next step was to obtain and adapt a mesh simplification algorithm to create meshes with managable triangle counts. This worked incredibly well – typical reductions were usually 50-fold. Additional techniques were applied after this to remove duplicate triangles, but that will be discussed in the next section.

 

 

Two days after starting, I’d written an alternate implementation of the world that attempted to render with a 3D camera. This was weird enough in itself – seeing TS1 graphics from another angle, even without objects, was really surreal. This mode immediately used a modified shader for drawing grass and build mode gridlines, but was completely unplayable. Ground and object clicking completely relied on the isometric perspective – and had to be re-implemented later in a more universal fashion.

Later that night, I got the game rendering objects too, and off the bat it ran surprisingly well. Some of the objects had clear z issues, and there was no way to render walls in 3d, but the game looked like it could potentially function at some point.

Objects quickly gained their own shader to function with our custom lighting, and using a simple bounding box + raycast I was able to make objects clickable as well.

Walls were quite tricky, since the original wall component generated a 2D sprite list, I had to re-implement wall rendering from scratch to generate 3D wall geometry with cuts and thick walls. This only took a day, but the recompile time for that single class is a testament to its complexity (though it is surprisingly simpler than the original 2D wall component).

One notable thing is that walls in The Sims 1 make no physical sense – no matter what direction you look at them from, the “front” of it appears to be dead on the edge of the tile, while the thickness of the wall extends back away from the camera. To represent this in 3D and avoid clipping issues with doors, walls and paintings, the wall actually has to shift away from the camera depending on the view direction. Thankfully this effect is subtle.

From this point, I added experimented with rendering the city view as part of the sandbox, and the result quickly became very cool. Around this point I also replaced our old mouse -> terrain position estimation with a more robust raycasting solution too, which allows objects to be placed in 3d, go here to work correctly and a few more core game features to work correctly. This change will actually also benefit the 2D engine – complex terrain will no longer be impossible to click on.

(these are development screenshots, the meshes for the objects, wall shad0ws and the skybox will look slightly different now)

This past week has been writing tweaks for the mesh reconstruction, playtesting, visual improvements, robust camera controls and 3D wall shadows.

Process

Any single graphical state for an object in The Sims is called a Drawgroup (DGRP). These contain information for positioning sprites in 3D space, for multiple zooms and rotations. We wish to create 3D meshes for each of these graphical states, which we can then render in place of the 2D sprites.

  • A heightmap-like mesh is generated directly from each sprite in the DGRP, and appropriately transformed from the isometric sprite (x, y, z) into object space. Meshes for “dynamic sprites” – toggleable sprites for each DGRP are kept as separate meshes to make them easier to toggle.
    • This can possibly be improved by omitting triangles that have already been “seen before”, and physically merging into the other sprites at exact vertices.
    • Some objects apply a post process effect to correct broken z values. This is discussed later.
  • The mesh is simplified using a quadratic mesh simplification technique. Our current implementation is a customized C# port of an existing C library – https://github.com/sp4cerat/Fast-Quadric-Mesh-Simplification . The port has been modified to prevent corner cutting, and to use a target precision rather than a target number of triangles.
  • The mesh is then cached for later use, and even saved to disk to prevent regeneration on next run.
  • This mesh can now be rendered alongside other 3D elements such as avatars, terrain and our new 3D walls, but with no limitations to camera position.

Can I run this on my Intel HD laptop?

Yes, but it will likely perform poorly and crash when loading any complicated lot (exhausted VRAM).

Most meshes for objects are generated and simplified algorithmically. This means that they will have a much higher triangle count than if the objects were modelled by an artist, mostly to avoid losing important details when simplifying the mesh. Typically, one tile of an object has several thousand triangles. This can quickly add up for thousands of objects, leading to millions of triangles potentially drawn in a given scene, and by extension millions of mesh vertex positions stored on the GPU.

Recommended Requirements?

You will need a gaming rig to run this smoothly with all features on. Here’s the machine I’m running on:

  • GPU: NVIDIA GeForce GTX 680
  • CPU: Intel i5-3570k @ 4.5Ghz

You’ll also need some free disk space to store cached meshes. Since the game does not use an mipmaps right now, I strongly recommend using the AntiAlias option toggleable in the options. This will apply 4xSSAA to the game graphics, which should be alright if your PC has a dedicated graphics card.

I can hit 144 fps in most lots, though in store lots and when seeing all objects in a lot (zoomed and panned camera) it can dip considerably. Other details are probably irrelevant – RAM usage is still within reasonable bounds. The game performs much faster in DirectX mode than it does in OpenGL, so I would recommend playing on DirectX when you are using 3D.

If you’re not targeting 144hz, even a laptop with a dedicated GPU should be able to hold 60. Mesh generation may take longer on slower CPUs though, and stuttering might be more present.

To run the game in 3D mode, toggle it on the launcher. If you’re starting the game directly from its executable, use the start up parameter -3d. This works for both FreeSO and Simitone. Make sure you have read this whole article though – our 3D mode will be near unplayable on hardware without dedicated graphics.

When the 3D update is available, changing this setting in the launcher will start the game in 3D mode.

Limitations

This technique works by assuming that the z-buffers for each object are an exact representation of the 3d shape of that object. A lot of the time this is not the case – the depth values have been modified for many objects to avoid them peeking through walls. Typically, this happens for counters.

A possible solution to this problem is to try and reconstruct most broken depth buffers automatically – detect if the object is affected (eg. assume affected if it’s a counter) and extrapolate the depth values to fix it before performing mesh reconstruction. This solution was implemented, since counters are such a common occurrence.

Automatic counter depth fix. X positions are extrapolated past a clip boundary based on the “border” vertices lying on this boundary. Keeps counters continuous.

Another issue is that the mesh reconstruction can only represents what has been seen from one of the four angles. This means that complicated meshes will have holes in them when seen from other angles, such as from a non-diagonal side or underneath. There can also be small holes between the tiles of multitile objects, since they are separate sprites in TS1. These issues are not as easy to solve.

Because it is an automatic process, and the z-buffers tend to have some error due to their limited byte precision, objects appear to have a “painterly” look up close, where they don’t appear quite as clean and straight cut as they do from the 2D isometric view. This is not a problem from a distance, but can be distracting for closeup shots.

Finally, some non-symmetrical objects have nonsense flips which were used to save space on sprites. For some of these flips I have simply disabled some of the sprites from other angles (typically windows), though this can cause holes to appear on the side of the object that does not have an accurate sprite.

One solution to all of these problems is to allow users to replace the meshes and textures the game generates with their own – allowing them to fix broken reconstructions, or even replace them with a better model with more detail using fewer triangles. This will be possible through Volcanic, our tool for editing and creating game objects, and one such replacement will be included with the game builds as an example.

An example process of exporting an ingame mesh, touching it up in Blender, and re-importing the fixed version of the mesh in the game.

I’ll leave these up to the community to manage. It’s possible to just clean up problem objects, but a larger scale project which remeshes every object in the game is a potential option too.

Summary

At the moment, our 3D mode is fully playable, but it should still be considered experimental, due to the presence of visual issues and low performance on integrated graphics. It’s a really surreal way to play the game that is definitely worth experiencing.

Most importantly for me, I’ve learned a lot developing this mode. This is likely one of the last things I will do full-time for FreeSO. Whatever I do next, whether it be developing my own simulation game or something less of a game, I’m glad I took the time to try this, and I hope you enjoy messing with it too.

Rhys.

28 thoughts on “The Impossible

  1. Awesome, thanks for everything and good luck on your next project. I know some people who’d love to play because they’re casual and only like 3D sims, and were turned off by isometrics.

  2. I just loved this, and also, i have to ask. Would this be possible as a mod to be developed to The Sims 1? I know you have a different system on running the game, so it’s probably a dumb question anyway.

  3. I’m happy to see that Simitone is getting the 3D update as well. I do hope that the game will be fully functional eventually because the idea of a 3D Magic Town is absolutely amazing (And so is the idea of a resolution larger than 800X600). This is amazing, I honestly didn’t think this was ever possible.

  4. I had thought that there could be a mode/option like this but it quickly passed through my head since it seemed unlikely that anyone would bother adding such a feature. Seeing the Sims in 3D is like watching the sims online trailer! Seriously, what a way to play FreeSO…….

    I am also very excited about being able to play the game(s) in higher resolutions with the GUI scaling.

  5. This is too surreal… This feels much like the original TSO trailer mixed with The Sims 2…. I am speechless. Thank you FreeSO devs, you’re my MVP.

  6. I think you could publish an IEEE paper on this. Just google “stereo reconstruction”. In a way our problem is simpler because we have a z-buffer for each angle (which the human eye does not) but I think we can still develop the mesh simplification algorithm further for the isometric case.

    1. There is definitely some possible ways to go for the reconstruction and simplification. The way it is right now is just the easiest way to make it playable and take the least amount of mesh generation time.

  7. Wow this is beyond amazing, thanks for all . If I may it would definitely be amazing to see more added to it and improvements but I understand if it wont happen … THANK YOU SO MUCH 🙂 From a sims lover since day 1

  8. Odd question, but Is it possible if SimCity 4 could be adapted to this program? Both The Sims and SimCity 4 I believe have a similar engine (They both probably have similar coding if I recall, but not sure) and they both use 3D meshes with an isometric camera view. I wonder if SimCity 4 could be able to become 3D like what you’re currently doing with The Sims and The Sims Online. Even though you’re main focus is primarily on The Sims/Online, just wondering if it could be done on SC4 which probably uses the same method of models or sprites just like The Sims.

  9. I’ve having a bit of an issue. I know that this project isn’t done yet but I just thought I should say that I’ve had an issue where the game starts to load and as it gets to “Reticulating Splines” it crashes. It throws up an error “CLR20r3”?

  10. please put a link somewhere on your page somewhere so people can donate to help your projects this is awesome =3 i know i’d love to donate

  11. Where can i donate? Never played Sims Online or EA Land, but i’ve heard stories about it over the time, and damn, this project needs support. If there would be more people like you…

Leave a Reply to Joseph Cancel reply

Your email address will not be published. Required fields are marked *