After last night’s flash snowstorm, weather forecasters in Sunrise Crater have been predicting persistent snowfall for the entirety of December, and a large part of January. Unfortunately, this means that residents may be unable to collect their annual christmas trees from the City Hall atop the mountain, and Christmas may be ruined for good. However! The Sunrise Crater government never backs down from a challenge, and has instead delivered all Christmas trees directly to your inboxes, free of charge. These special trees come retrofitted with (refilling???) presents beneath them – 3 of which are sweet treats, and one of which is a special gift for each day… You’ll have to find out for yourself what’s in each box!
Festvities aren’t the only thing we’re bringing to the table this update. Though not much development work has been going on from my side, our community and other developers have been pushing to push the game further to being functionally complete, and even beyond. Here’s a list of all the stuff we have going now, newest first:
Intoducing Motive Overfill
Visiting some types of lots will now allow you to fill your motives for a lot longer than other lots. These “overfills” will persist as you move across lots, so are perfect before you run a job, group money, single money or skill. To obtain overfills, go visit a lot from that type and green up in their overfill category. If a lot doesn’t exist for that overfill category, why not make it yourself?
Here’s a list of the tunings we’re demoing this feature with:
Service Lot: 50% overfill on hunger, energy, hygiene and bladder.
Entertainment Lot: 100% overfill on fun and social.
Romance: 100% overfill on social. (pending additional feature)
Note that this is not all we have planned for other lot types. Several additional systems are in the works to make these categories more appealing or have more of a purpose.
Custom Content within official release
A full serverside infrastructure for uploading, managing and selling custom content may be a long way off, but that hasn’t stopped our community from getting their hands dirty! Using our object development tool, Volcanic, not only have we patched new functionality and translations into existing objects, but users have already created entirely new objects from scratch. These include entirely new slot machines with custom UI graphics (and a themed painting for the newest one!), as well as objects that you might find under your Christmas tree…
This Halloween, I created an NPC with Volcanic which would serve as a proof of concept for the kind of complex NPC and object functionality we could potentially add to the game. This mainly existed for fun, but the object itself was promising – the skeleton drove its own AI for detecting certain classes of game object and performing interactions with them, without using the normal Free Will system which limits sims to doing rather normal actions. More complex creations could be coming in the future – either from me, or anyone else willing to try out SimAntics scripting.
3D Remeshing Team
In the last post, I outlined some issues with our automated mesh generation that you probably noticed yourself looking at the video or playing the game. These meshes typically also consisted of many different parts with textures from different rotations, so they take many more draw calls to render than models in typical 3D games. One of the solutions to this problem was for users to recreate the meshes of the objects in the game from scratch, either by using the game sprites to supply texture to their custom meshes or by drawing their own textures from scratch.
The response from the community has been fantastic, multiple users have started remeshing game objects that the reconstruction didn’t do justice – sometimes even entire object sets. Object reconstruction is usually discussed in our Discord server, where we’re more than happy to help you get started with meshing if you’re interested in helping.
Roulette Table Plugin
Developer The Architect has struck again, with an incredibly clean implementation of one of TSO’s most complex plugins. This object lets you play roulette with a few other sims at entertainment lots, and gives casinos a lot of character. Bet you can’t wait to see what he has next in the pipeline! :scream:
Simitone for Windows Updates
If you’ve been keeping track on the discord, Simitone for Windows has recently updated to support Build and Buy mode (including simless build/buy) and a work in progress CAS (can’t save families). In a short while the game should also support saving, though it might still be fun to check out if you want to mess around with building TS1 lots in 3D. (note that builds of the game will never be available on iOS)
Advanced Shadows using 3D Models
3D mode was not only developed for the promise of playing TSO in 3D – if the reconstruction proved to be inaccurate or too ugly for the game to be playable in that state, then the 3D meshes could be used for another purpose – detailed sun and point shadows. The results are really cool, and people have been really putting the lighting to use giving their lots some real atmosphere.
The advanced lighting works in 2D and 3D, but the clean look you get in 2D is rather hard to beat here. Maybe with more remeshed objects, 3D can come close to comparing with the original 2D sprites.
FreeSO for Windows… on mac
A few weeks ago, I discovered that it is actually possible to run our windows builds on mac, so long as you aren’t afraid of using the terminal to run Mono, and manually extracting some replacement libraries. However, the solution works even with game updates, which means that once you set it up you’re good to go!
Note that this is not an official mac release, this is simply a rather complex way of getting the Windows build of the game to work on another operating system by utilising Mono. The game now also works on linux, though I haven’t tested a methodology to that myself.
With this update to FreeSO, we have enabled two new experimental features. The first is a transaction logging system, which should log recent transactions from payout objects and from sim to sim, for analytic purposes. The second is a dynamic tuning system, which pulls tuning values from our database when lots are opened, and sends them out to players without them having to update their client. This second feature is currently mainly being used for motive overfill tuning, though it can actually be used for any tuning value in any existing object, including payouts for single and group money objects. We can even set up a system to dynamically vary tuning like this based on the transaction data we have collected, making less appealing skills pay out more money to make up for it. After we’ve collected some data, you should see this system rolling out, alongside more quality of life changes for unpopular lot types. Stay tuned!
EDIT: The update has been available since last Friday. You can check it out by downloading the latest update from FreeSO servers, then following the instructions later in this post!
Three years ago, any chance of anyone reviving The Sims Online looked very slim. Any existing project seemed to either fall off in activity, be a hoax in the first place, or burn hopeful fans with controversial donations for a server that was never meant to be. Despite all the ruckus, three years ago is when we got the first glimmer of hope that TSO would be back again – a convincing UI recreation of the initial few screens of the game including Create A Sim, with a lot of backend work to load important game content. Developed by Afr0 and ddfczm, this recreation was called Project Dollhouse, and it’s what inspired me to work towards this same goal. Despite exciting developments in SimAntics emulation by me and ddfczm, with ingame footage of TSO objects somewhat working, this community stayed at only a few members – nobody believed anymore in TSO ever returning. Our community slowly grew as I implemented the online sandbox mode and split into FreeSO, and people finally started believing that it was possible. The launch into beta was a spectacular surprise.
Today, we’ve almost done it. It has been a heck of a long journey, and finally people don’t have to believe in something anymore – it’s right here. A fully re-implemented TSO client and server is available for all to use, with source available so it can be developed further by anyone in the future, maybe tens of years from now. The ability to play TSO will never truly disappear, thanks to the efforts of its community.
Recently, I attempted to port The Sims 1 to mobile by myself, something many users never would have believed would work until recently. Unfortunately this was not meant to be, but I demonstrated that it was never impossible, and that it can even be done well by an independent developer.
To keep up with my trend of doing stupid things nobody believes in, I decided to make good on a crazy idea I shared three years ago, and have even mocked on one april fools since it felt so unfeasible at the time. I had an idea that The Sims and The Sims Online could potentially be rendered and playable in 3D, in real time, using mesh reconstruction techniques. Our next update will have an experimental mode showing that this is indeed possible!
I’m guessing your immediate thoughts from seeing our header image is, “how is this even remotely possible? the objects in this game are 2D sprites”. This article will also explain how it wasn’t as impossible as it seemed, and explain how it was achieved.
Update timing and changes
The update will release within the week. It will include the 3D mode shown in the video (see later in this post for more info), but also several critical bug fixes including fixes for interactions, disappearing purchased objects, lot save mishaps and many common SimAntics exceptions.
Note that in 3D mode, the first time you see each drawgroup the game will freeze or stutter heavily. This is because it is generating the 3D meshes the game uses from the sprites – these are not included with the FreeSO client!
The update also includes two enhancements for 2D mode. You can now use the mouse wheel to zoom in and out, and our new 3D wall component can be used in 2D mode to generate 3D wall shadows. You’ll find a toggle for this in the options menu.
3D in Simitone for Windows
Though Simitone can no longer be developed for mobile devices, the source is still available and buildable on windows desktop. Since Simitone runs on the FreeSO engine, these changes work in it as well. Gameplay is still in the same state as seen in the last video, so it’s not entirely playable.
The idea of converting TS1/TSO objects to 3D was actually something I was interested in even before I started working on Project Dollhouse or FreeSO. While Niotso and PD were still active, I frequented the Niotso IRC on hackint, where we discussed the various formats in the game.
Most other isometric games simply draw sprites from back to front, which works well for most games, assuming that the sprites are never meant to intersect each other. The Sims is a rather special case, since object sprites intersecting with avatars is a key requirement for most of the objects – specific parts of a shower, for example, must appear over the sim while they are in it. For this purpose, all object sprites in The Sims have a little extra depth… in that they literally store a depth value for each pixel in the sprite, as it was when it was originally rendered from a 3D model. This is called its z-buffer, and allows per pixel depth comparison between objects and avatars.
This was not the only function of this renderer, though. A custom version I made was actually able to render objects from non-standard angles, using the z buffers and some vector math to offset their pixels to a new position in 3D space. (press 1, 2 to rotate)
This was a good proof of concept, but obviously this kind of 2D per-pixel transformation is too slow to run in real-time on a lot. It was clear a true solution would need to generate a 3D model which could be transformed and rendered on the GPU.
You could say that this has always been the secondary goal of this project, though obviously I never like to get people’s expectations up if they might be disappointed. It’s always been on my mind, either way.
Developing the whole thing into a playable state took about a week. Not kidding – since the entire game was already there, and terrain was already in 3D, it didn’t take long to get a prototype working. The first steps were simply generating meshes for existing objects in the game and displaying them by themselves in volcanic. This was done to test how viable applying such a technique to the whole game might be, on a per object basis.
These triangle meshes were literally generated as heightmaps of the z-buffers, without any tricks. It became clear that these meshes would have too many triangles to render 1000-fold on a typical TSO lot, so the next step was to obtain and adapt a mesh simplification algorithm to create meshes with managable triangle counts. This worked incredibly well – typical reductions were usually 50-fold. Additional techniques were applied after this to remove duplicate triangles, but that will be discussed in the next section.
Two days after starting, I’d written an alternate implementation of the world that attempted to render with a 3D camera. This was weird enough in itself – seeing TS1 graphics from another angle, even without objects, was really surreal. This mode immediately used a modified shader for drawing grass and build mode gridlines, but was completely unplayable. Ground and object clicking completely relied on the isometric perspective – and had to be re-implemented later in a more universal fashion.
Later that night, I got the game rendering objects too, and off the bat it ran surprisingly well. Some of the objects had clear z issues, and there was no way to render walls in 3d, but the game looked like it could potentially function at some point.
Objects quickly gained their own shader to function with our custom lighting, and using a simple bounding box + raycast I was able to make objects clickable as well.
Walls were quite tricky, since the original wall component generated a 2D sprite list, I had to re-implement wall rendering from scratch to generate 3D wall geometry with cuts and thick walls. This only took a day, but the recompile time for that single class is a testament to its complexity (though it is surprisingly simpler than the original 2D wall component).
One notable thing is that walls in The Sims 1 make no physical sense – no matter what direction you look at them from, the “front” of it appears to be dead on the edge of the tile, while the thickness of the wall extends back away from the camera. To represent this in 3D and avoid clipping issues with doors, walls and paintings, the wall actually has to shift away from the camera depending on the view direction. Thankfully this effect is subtle.
From this point, I added experimented with rendering the city view as part of the sandbox, and the result quickly became very cool. Around this point I also replaced our old mouse -> terrain position estimation with a more robust raycasting solution too, which allows objects to be placed in 3d, go here to work correctly and a few more core game features to work correctly. This change will actually also benefit the 2D engine – complex terrain will no longer be impossible to click on.
(these are development screenshots, the meshes for the objects, wall shad0ws and the skybox will look slightly different now)
This past week has been writing tweaks for the mesh reconstruction, playtesting, visual improvements, robust camera controls and 3D wall shadows.
Any single graphical state for an object in The Sims is called a Drawgroup (DGRP). These contain information for positioning sprites in 3D space, for multiple zooms and rotations. We wish to create 3D meshes for each of these graphical states, which we can then render in place of the 2D sprites.
A heightmap-like mesh is generated directly from each sprite in the DGRP, and appropriately transformed from the isometric sprite (x, y, z) into object space. Meshes for “dynamic sprites” – toggleable sprites for each DGRP are kept as separate meshes to make them easier to toggle.
This can possibly be improved by omitting triangles that have already been “seen before”, and physically merging into the other sprites at exact vertices.
Some objects apply a post process effect to correct broken z values. This is discussed later.
The mesh is simplified using a quadratic mesh simplification technique. Our current implementation is a customized C# port of an existing C library – https://github.com/sp4cerat/Fast-Quadric-Mesh-Simplification . The port has been modified to prevent corner cutting, and to use a target precision rather than a target number of triangles.
The mesh is then cached for later use, and even saved to disk to prevent regeneration on next run.
This mesh can now be rendered alongside other 3D elements such as avatars, terrain and our new 3D walls, but with no limitations to camera position.
Can I run this on my Intel HD laptop?
Yes, but it will likely perform poorly and crash when loading any complicated lot (exhausted VRAM).
Most meshes for objects are generated and simplified algorithmically. This means that they will have a much higher triangle count than if the objects were modelled by an artist, mostly to avoid losing important details when simplifying the mesh. Typically, one tile of an object has several thousand triangles. This can quickly add up for thousands of objects, leading to millions of triangles potentially drawn in a given scene, and by extension millions of mesh vertex positions stored on the GPU.
You will need a gaming rig to run this smoothly with all features on. Here’s the machine I’m running on:
GPU: NVIDIA GeForce GTX 680
CPU: Intel i5-3570k @ 4.5Ghz
You’ll also need some free disk space to store cached meshes. Since the game does not use an mipmaps right now, I strongly recommend using the AntiAlias option toggleable in the options. This will apply 4xSSAA to the game graphics, which should be alright if your PC has a dedicated graphics card.
I can hit 144 fps in most lots, though in store lots and when seeing all objects in a lot (zoomed and panned camera) it can dip considerably. Other details are probably irrelevant – RAM usage is still within reasonable bounds. The game performs much faster in DirectX mode than it does in OpenGL, so I would recommend playing on DirectX when you are using 3D.
If you’re not targeting 144hz, even a laptop with a dedicated GPU should be able to hold 60. Mesh generation may take longer on slower CPUs though, and stuttering might be more present.
To run the game in 3D mode, toggle it on the launcher. If you’re starting the game directly from its executable, use the start up parameter -3d. This works for both FreeSO and Simitone. Make sure you have read this whole article though – our 3D mode will be near unplayable on hardware without dedicated graphics.
This technique works by assuming that the z-buffers for each object are an exact representation of the 3d shape of that object. A lot of the time this is not the case – the depth values have been modified for many objects to avoid them peeking through walls. Typically, this happens for counters.
A possible solution to this problem is to try and reconstruct most broken depth buffers automatically – detect if the object is affected (eg. assume affected if it’s a counter) and extrapolate the depth values to fix it before performing mesh reconstruction. This solution was implemented, since counters are such a common occurrence.
Another issue is that the mesh reconstruction can only represents what has been seen from one of the four angles. This means that complicated meshes will have holes in them when seen from other angles, such as from a non-diagonal side or underneath. There can also be small holes between the tiles of multitile objects, since they are separate sprites in TS1. These issues are not as easy to solve.
Because it is an automatic process, and the z-buffers tend to have some error due to their limited byte precision, objects appear to have a “painterly” look up close, where they don’t appear quite as clean and straight cut as they do from the 2D isometric view. This is not a problem from a distance, but can be distracting for closeup shots.
Finally, some non-symmetrical objects have nonsense flips which were used to save space on sprites. For some of these flips I have simply disabled some of the sprites from other angles (typically windows), though this can cause holes to appear on the side of the object that does not have an accurate sprite.
One solution to all of these problems is to allow users to replace the meshes and textures the game generates with their own – allowing them to fix broken reconstructions, or even replace them with a better model with more detail using fewer triangles. This will be possible through Volcanic, our tool for editing and creating game objects, and one such replacement will be included with the game builds as an example.
I’ll leave these up to the community to manage. It’s possible to just clean up problem objects, but a larger scale project which remeshes every object in the game is a potential option too.
At the moment, our 3D mode is fully playable, but it should still be considered experimental, due to the presence of visual issues and low performance on integrated graphics. It’s a really surreal way to play the game that is definitely worth experiencing.
Most importantly for me, I’ve learned a lot developing this mode. This is likely one of the last things I will do full-time for FreeSO. Whatever I do next, whether it be developing my own simulation game or something less of a game, I’m glad I took the time to try this, and I hope you enjoy messing with it too.
In the last news update, I announced an attempt at using the FreeSO engine to bring TS1 to mobile platforms. For the foreseeable future, these efforts have been stopped.
A few months ago we were contacted by EA regarding how the TSO IP should be handled. At that time, you may have noticed a change in logo to avoid the house motif, the addition of multiple disclaimers to our websites, and further insistence on taking no donations for server costs (really, no change). Given even just its relevance to me, The Sims’ IP is clearly something that is worth EA protecting from any potential damage, so I complied with these terms immediately out of respect.
A few days ago, some time after our last blog post, I received further correspondence from EA, regarding my plans to bring TS1 to mobile devices, as well as potentially FreeSO. To protect their IP, they asked that I cease and desist any efforts to bring either of these games to mobile or any other plaforms. While both projects are entirely legal (copyrighted content and references to “The Sims” provided by the user, not the replacement engines themselves), I do not want to step on EA’s toes, and will obviously comply with their requests rather than starting some kind of expensive legal battle.
Seeing it first hand, I could completely understand how damaging a third party releasing a mobile client could be to The Sims brand. The mobile market had never seen a full The Sims game with no micro-transactions, and even though it might be hard to install and transfer the game from a computer (especially on iOS), the disruption could have had a notable impact.
Granted, only a few weeks were spent on the project, but it’s unfortunate that after seeing it work so well, it may never see the public in a complete fashion. I enjoyed working on the project, and it taught me a lot about mobile development with Xamarin & MonoGame, multi-touch interfaces and more about The Sims (and how it differs from TSO).
Despite the fact that I cannot release it, what I created is really worth seeing, and I wouldn’t want people to think I was lying about the whole thing. Therefore, I’ve prepared a video created with the final iOS build I produced, showing all implemented features in action (running on iPhone SE):
You will notice many bugs. You will notice that the game sometimes stutters, especially at three speed. All of these issues are caused by lack of optimisation – with some amount of work the game could run smooth as butter at all times on devices such as the iPhone SE and 6. It’s a great proof of concept for how the FreeSO engine was extensible enough to quickly run TS1’s game objects and logic with a flexibility not possible with the original game engine. I’m personally fine with simply showing such an achievement was possible.
The last version did not have build/buy, the fame system, save games and many other things, so it was far from playable.
Simitone’s fallback purpose
Though I cannot release mobile projects, I can release what I have as an alternative Windows client. This client is currently very useful for examining how TS1 objects work. In the future, it could improve the playability of the game on high resolution monitors.
The Simitone replacement game client has therefore been released on Github for use on Windows Only. This is not open source in the same vein as FreeSO – the Simitone frontend is entirely a personal project, and has all rights reserved… for now. FreeSO is a subproject of Simitone, which all sources and rights are still available for. The FreeSO engine actually contains most of the parts of , as you can see through its hybrid mode.
I was working on the TS1 port because it was something I enjoyed doing. I quickly became passionate about finding the right ways to handle the UI for the game on mobile devices. While I am alright with this outcome, getting knocked out of working on it has caused me to burn out a little.
The reality is that I cannot keep working on this project forever. Over the past three years, I have hoped that I would eventually be able to pass development onto a community of developers, and then progress to do my own thing. Despite some individual developers implementing crucial game features, (ddfczm long term, and The Architect for some recent object plugins) this community of developers has not materialised. The truth is, finding developers for an open source MMO server is incredibly difficult for a number of obvious factors: we need a ton of experience, a self-driven attitude and dedication to TSO. These things do not seem to overlap very often.
I am currently working on this project while unemployed. While I do enjoy it, neither leeching off of others or starving to death are the best I can do with my life. I wouldn’t take donations for ethical and personal reasons anyways, let alone potential legal issues.
For now, I will be taking a break for a few weeks to figure things out, after I push some fixes for stuck lots, the killer Mina.NET bug and a fix for some error 23s.
Over the past few weeks, I’ve been working on getting The Sims 1 working within our engine, to ready it for a mobile version. This has led to the development and improvement of a feature that was not explored in TSO before: terrain tools.
This enables a whole new level of creative freedom for designing interesting lots which can use the terrain to spice up a lot with multiple buildings, or even enable lots based entirely around a terrain feature such as a mountain, mesa or digsite. I for one would like to see a jam lot in a digsite, so sims can bury their dreams while they make jams just to save some time.
Immediately, you’ll remember that in the original TSO terrain tools were always listed as “Coming Soon!” in the build mode panel. The only place the terrain really appeared was in the very edge of the lot, where it only slightly resembled the terrain around your lot. Now when you place a lot on uneven ground, the terrain will accurately reflect what you see in the map view, including steep cliffs.
Additionally, floor tiles and foliage can now be placed on slopes, thanks to a new FSO custom 3D floor renderer. Terrain was very limited in TS1, so it was only really used for one lot in Magic Town, where they even had trouble making a path down the hill due to sloped floor tiles being impossible.
The tools themselves have been greatly improved from TS1’s versions. In the original, you had to click, follow and hold on a tile corner to raise or lower it, leading to you occasionally raising the wrong corner and generally having little precision or control over the results. In our implementation, raising and lowering terrain is possible though a tool similar to that present in Roller Coaster Tycoon, where you click a vertex and drag it to its target height. Additionally, you can hold the control key to provide a smoother falloff rather than the default “pyramid” pattern.
The flatten tool now works on a vertex level rather than a tile level. Select a terrain vertex with the target height, and simply drag a rectangle out over the vertices you want to be the same height. You can hold the control key to produce a smoother falloff here too.
I have also re-implemented the grass tool, which works like a paintbrush. To remove grass you hold control, and to change the grass state more gradually you can hold shift. Mousing over an area once changes the grass state by one level – mousing over it again or reclicking on the point will change it by another.
A work in progress
Note that while the feature is mostly complete, it has been released early in order to deploy emergency changes which prevent server crashes. This means that a few things may not look perfect, and due to the complete overhaul of the floor rendering, some things may be missing. This includes artefacts in water/pool rendering, diagonal tiles, and missing corner pieces on pools.
Additionally, you might find it hard to click on terrain that is high enough to cover other terrain in the background (go here and build/buy placement). This is because we must implement a new way to detect which tile the mouse cursor is over.
The Sims 1 in FSO
Since The Sims Online is actually built on the engine of The Sims 1, it’s not too much of a leap to adapt our engine to support and use objects and resources from the original game. After quite a bit of work, we are now running nearly all lots in a default TS1 neighbourhood (including downtown, studio etc), without much issue. You can imagine a few benefits to playing TS1 in the FSO engine, such as the lighting, high framerate support, 5 floors and powerful modding tools… but the true final objective is to get The Sims 1 running on mobile devices with an entirely custom UI.
While this is mainly a project in its own right, the development of this engine is gradually improving the state of our TSO engine. I’ve tracked down multiple very specific bugs which were completely untracable within TSO’s objects, notably the “running man” interaction bug and obscure behaviour with animation events. It was the development of this project that lead to the addition of terrain tools across both projects.
Finally, here is a mockup of the mobile friendly UI that will be used in the TS1 frontend. Note that these are just a few shots out of a large collection of menu mockups, but you should be able to note that we’re keeping it simple.
It’s been a long time since our last update, but I’ve finally got some time on my hands to get back into things. We also have a new developer working on object plugins for the game too, which have been gradually been appearing in the game since my last post.
The main improvement in this patch is something I didn’t expect to be completed this early into the process, and I didn’t think it would ever be this beautiful and effective. This patch introduces a completely custom lighting engine, built from the ground up for TSO/TS1.
Even though it is not a necessary feature, it’s something I’ve had on my mind for an incredibly long time. Sims games past 1 typically build a light map to construct smooth falloffs and shadows for light sources, with 3 and 4 actually featuring advanced lighting which models per-light shadows in each room. The goal was to bring these later developments back to TSO, now that we have the graphics hardware to pull it off. Wall shadows are projected from each wall “line” in the light’s room, while object shadows are estimated from the object’s collision bounds (both with smooth falloffs).
Around a month ago, the owners of the longstanding legacy server freeso.ml and I decided to collaboratively set up a test server for people to play in, which would inevitably be wiped as game breaking economy and skilling bugs were worked out. This was purely so people could play the game in its current, mostly working state, rather than waiting for me to deal with my RL business before they could play at all. Our hosts and moderators have proven an absolute blessing in that I’ve been able to put the right amount of time into everything, with them handling support and keeping the server alive.
The server was advertised on the discord and partially the forums, so that the most dedicated users could have a chance at playing first. These aren’t our only users though – we’ve received a lot of attention from variousstreamers over the past few weeks, which has set off a few communities, top 100 lot competition. At max, we’ve sustained 350 concurrent players in around 60 lots, on one server box. There are just over 2000 lots on the map, and 11,521 avatars in existence. Cumulatively, sims have §75,528,319 in the bank, and have bought §92,393,467 worth of objects (191,956 objects)!
Though early testing, it’s clear the transition to ASP.NET will work. While Nancy would crash with a pitifully low number of requests, ASP.NET does not drop any requests at all with several thousand. However, as I mentioned before, we did not expect to see several thousand users attempt to register on day one. Admittedly, we got a bit ahead of ourselves; planning to implement many moderation features after the game had been established, due to a rather small expected userbase.
Ouch! When we tried to launch the game tonight, the servers literally couldn’t take the insane load that we underwent. Here’s everything that happened:
One day before launch, DOS attacks began hitting the wordpress and forum servers. These were mitigated as they were pretty small scale, and against normal web servers which can deal with a reduced load much better.
When setting up the API server on our large scale servers, DOS attacks began coming through to the API server recently added to the DNS. These were rate limited, but somehow were still enough to take down the server. It became clear something was wrong with the Nancy driven API server from this.
By rate limiting to 10 requests a minute (yikes!) we were able to reduce the load on the registration enough to keep it alive, before it was even announced to anyone. During this period, I was able to log in with what seemed like no issues. In this period, 100 users registered.
I released the registration link on the discord channel. The whole thing blew up, immediately 500 concurrent requests were made to the api server, taking it down immediatly. Hours later, these have still not simmered down – there are a few hundred requests being made and simultaneously active. The large delay each of these requests is incurring is causing the majority of them to fail to get through.
Throughout this process, it became clear that there was a serious issue with the amount of load the API server was able to take. I would say that it should have been able to take at least 25x as much load, and it certainly should not have repeatedly crashed. After some quick experimentation tonight, it seems that the HTTP serving library used to create the API server, Nancy, was serving even plain non-sql requests way slower than other libraries.
We also did not expect this many registrations. Of all of the requests, 885 users from unique IPs signed up. This makes things really difficult. I’m only one developer, and the flash interest in a game that basically died of inactivity years ago is very surprising. It’s also pretty satisfying; it is fantastic to see this much interest in FreeSO, it’s just also very hard to manage.
Could this have been mitigated by opening registrations earlier? Could users whose registrations got through get to play the game?
No, the API server does not only drive registration. If we were to somehow mitigate impact on registration, then the impact caused by users logging into the game would bring it down instead. This would be a lot more certain, as authentication makes 4 times the requests as registration. Registration was also not ready until close to the deadline. This is mainly due to the DOS attack to the wordpress mentioned before.
So what is the plan?
We will perform further load and api testing, look into the root causes of load issues and the server stability. The current plan is to rewrite the API server using ASP.NET instead of Nancy, as it is a lot more “battle-tested”. This could take some amount of time.
We cannot announce a retry date. We might even need a new launch plan altogther, to avoid the load of a few thousand users wanting to play the game on day one. Whatever the plan is, it will be posted on here eventually.
The first city the official FreeSO server will feature is the evolution of Test Center, Sunrise Crater!
I originally planned to reveal this on December 25th, but I got a little carried away with the changes and town layouts. The city is now full of small towns and interesting features for communities to spring up around. Each part of the city has been intricately detailed by hand, from within the client, rather than just whipping up some graphics in a standard editor. Thanks to this, the difference is so staggering you might be surprised that this is still using the same landscaping system as the original.
It has been a very long journey. Started on February 16th 2014, the SimAntics & World Development thread was the first attempt at re-implementing the complex Virtual Machine behind both The Sims and The Sims Online. 17 months later on July 28th 2015, we ran our first online playtest, where everyone appeared as Bob Newbie, the server crashed every 5 minutes and many tables were stood within. Now, 17 months later again, for New Year 2017, we’re finally launching the game in the large scale MMO context that we’ve been working towards this whole time.