[RM-4] TCP Sockets in RPG Maker VX

Blog Info: I was in a bit of a slump lately for new post ideas –it seemed that all my good ideas were far too big, and I didn’t really want to get side-tracked like I did for the EL client notifier. So, I decided to look into the possibility of doing networked games using RPG Maker. Along the way, I finally found a good way to highlight code samples on this blog –by using Scite’s “Export to HTML” feature!

Back-story: When RPG Maker VX came out, it radically changed the organization of Ruby code internal to the engine. What it did not change was the fundamentals of the scripting system itself. Thus, any code in RPG Maker XP could feasibly run on VX –assuming that it made no reference to windows, sprites, etc. Unfortunately, porting this code was not always easy. One module that never made the transition was the RPG Maker XP Online System, programmed by username “Blizzard”. This system was a beautifully designed, fully-functional API for RPG Maker network play. Fortunately, Blizzard released the source code for this engine under the Creative Commons (NC-SA) license. This first entry will deal with the simplest –but most important– problem: using sockets. After that, we will slowly build up.

Goal: Create a simple proof-of-concept that TCP sockets can work on RPG Maker VX.

Table of Contents

Step 1: Use Case

Step 2: Borrowing Library Code

Step 3: Initializing & Testing the Connection with NPCs

Step 4: Taking a Step Back

Step 5: A Slightly More Impressive Proof-of-concept

Game Prototype 1: Autosave and an IP NPC

Game Prototype 2: An Online Community

Game Prototype 3 – Chatting with Other Players

Further Work

Step 1: Use Case

Let’s start with a brand-new RPG Maker project. Create a starting map, and put down some “land” tiles. Then, add an NPC with the following commands:

• Message Box: “Opening Socket”
• Script:

s = TCPSocket.open('127.0.0.1', 7689)
s.close()

• Message Box: “Success!”

Now, start your game and talk to your NPC. You should see the first message box, and then the game will crash.

No "sockets" library means no TCPSocket class, which in turn means that our two-line script will crash horribly.

What just happened? Well, first we tried to create a socket. This socket assumed that we have a server running on 127.0.0.1 (which means “this computer”) and that it is bound on port 7689. Since no server has been started, we expect a crash –however, we get the wrong kind of crash. If you look at the message box, you’ll see that RMVX had trouble finding the “TCPSocket” item. In other words, RPG Maker VX does not ship the default Ruby networking libraries! Oh no!

The thing is, RPG Maker XP also didn’t ship with networking support. So how does RMX-OS use it? If you’ve studied programming languages, you know that features such as networking are not considered part of the language itself; they’re usually linked up directly to the operating system after the language has been created. Doing something like that manually is far too difficult for the average programmer. Fortunately, we might be able to “borrow” the code that Ruby itself uses to connect to sockets.

If you want to follow along, you can download Ruby’s source (it must be version 1.8) from some online source –I recommend the Debian repository, with links below. However, the source really isn’t necessary, since I’ll be quoting samples from it when you need them. Moreover, I can’t seem to find the relevant socket code (although I did find useful Mutex code), so you probably don’t need to download the source unless you’re curious. Here are the links:

http://ftp.de.debian.org/debian/pool/main/r/ruby1.8/ruby1.8_1.8.7.249.orig.tar.gz

…or, more generally, from:

http://packages.debian.org/en/squeeze/ruby1.8

After downloading it, unzip it and remember where you saved the source. The only file we’ll need is thread.rb.

Step 2: Borrowing Library Code

One thing you may have noticed if you’re familiar with ruby is that RGSS (RPG Maker’s Ruby scripting system) doesn’t support the “require” primitive. This makes sense –since “require” searches the standard library path,  you might accidentally load Ruby library code which your playtesters likely won’t have. That said, RPG Maker is still perfectly capable of running native code from DLLs. For example, the following works just fine:

Win32API.new(DLL, 'connect', 'ppl', 'l').call(1, 2, 3)

Thus, we might consider chaining together DLL calls to create a kind of “fake” networking library. Eventually, we’ll probably try to migrate to a so-called “native” library, so that we can gain the benefits of low-level primitives. An example is provided in the win32utils:

http://rubyforge.org/projects/win32utils/

If you download and scan, say, the win32-nio library source, you’ll see lots of “require” tags at the top of the main library files. Be aware that you’ll have to essentially replace these with compatible functionality if you want to use the library.

Therefore, we won’t use win32-nio just yet. Instead, we’ll copy the Win32 API that “blizzard” put inside RMX-OS. He claims that it’s from Ruby’s 1.8.1 library source, but for the life of me I can’t find it online. That’s not really a problem, though, as the Ruby License allows verbatim copies of code and distribution in some format.

RMX-OS’s networking code is, by the way, built on top of DLL calls. Here’s the code you need to copy:

Code Segment 1 — Apologies for the ODT format, but PDF removed newlines and tabs.

In the Script Editor (F11), right-click on “Main” and choose “Insert”. Type “Ruby Library Code” in the “Name” box, and then paste the source code from segment 1 into the edit panel. Choose “ok”, then save your game.

If you run your code now, you’ll get some vague, disheartening runtime error. So, before you get depressed, delete your test NPC and add the following code directly into the Main script, below Graphics.freeze:

s = TCPSocket.open('127.0.0.1', 7689)
s.close()

Now, save and run the game. You’ll get the following error on startup:

This error means the library code loaded, then couldn't find the server. We are making progress!

Now that’s the kind of error we’re looking for! Congratulations, the basic structure of the TCP code you added was correct!

Step 3: Initializing & Testing the Connection with NPCs

We need to test an actual TCP connection –in addition, we should probably fix that bug where NPCs can’t seem to trigger our code. These are our next tasks.

If you try to track down the error from the NPC-enabled network code, you’ll reach the following line:

  def self.check
    errno = Winsock.WSAGetLastError
    raise Errno.const_get(Errno.constants.detect { |c| Errno.const_get(c).new.errno== errno })
  end

The error is located inside that particularly nasy “raise” statement. If you don’t understand Ruby’s “block” mechanism, you’ll probably feel like quitting right here. Instead, try this simple abstraction:

  def self.check
    errno = Winsock.WSAGetLastError
    constName = Errno.constants.detect { |c| Errno.const_get(c).new.errno== errno }
    raise Errno.const_get(constName)
  end

Run this code again, and the error’s still on the “raise” line… but now it’s superficially easy to understand. The original programmer was trying to look up Win32 error messages by ID, and he didn’t account for the possibility that a null socket might be returned with NO error code being generated in the process. Well, this is easy enough to fix:

  def self.check
      errno = Winsock.WSAGetLastError
      constName = Errno.constants.detect {|c| Errno.const_get(c).new.errno == errno }
      if constName
        raise Errno.const_get(constName)
      else
        raise "Unknown network error code: #{errno}"
      end
  end

Now, if you try to trigger a TCP connection from the NPC, you’ll see your “Unknown network error code” message.

At this point, I should probably mention that using TCPSocket.open() is not technically correct, since it expects to be passed a block of code to execute upon completion. Although that might be ok for simple socket-related tasks (like requesting the current time or weather conditions from a central server), we’ll need a persistent connection for our game engine. As you might expect, our code will overflow the limits of an NPC’s script command. So, change the NPC’s script code to read:

tcptest()

…and add the following to the “Main” module, directly above the first “begin” statement:

  def tcptest
    #Createa a socket
    s = TCPSocket.new('127.0.0.1', 7689)

    #Send a test message
    s.send("Testing...\n")
    #Receive a result from the server
    msg = ''
    while
      buffer = s.recv(1024)    #Read UP TO 1024 bytes
      buffer.gsub!(0.chr, '') #Remove null bytes
      msg += buffer            #Append received data
      break if buffer.count("\n")>0   #Stop if we've reached the newline
    end
    #Done; close the socket, print our message
    s.close()
    print "Received: #{msg}"
  end

Our code is fairly simple; we use “new” instead of “open”, and then rely on “send” and “recv” to send and receive our data. One problem with “recv” is that it won’t wait until the server has sent the entire message, so we have to chain together all of the bits and pieces of the message as they arrive. On the upside, one of the guarantees of TCP is that we’ll never receive messages out-of-order, so that’s one less potential headache.

Now all we need is the server. I find Ruby a bit distasteful for client/server code, so let’s try writing one in Java. One issue with Java is that compiling and running it can be a pain. Rather than give you a full tutorial on Eclipse or javac, I’m going to ask you to download and install TextPad.

Edit: Some users reported needing to download the JDK, too. Open a command prompt and type javac -version. If you get an error message, you’ll need to download the JDK. If all else fails, see the “source code” section for compiled versions of the server code.

Now, create a new file, and save it as SimpleServer.java. The name must be spelled exactly this way; such a naming convention is a feature of Java. Inside the file, add the following code:

import java.io.*;
import java.net.*;

class SimpleServer {
  public static void main(String argv[]) throws Exception {
    String clientSentence;
    String capitalizedSentence;
    ServerSocket welcomeSocket = new ServerSocket(7689);

    for ( ; ; ) {
      System.out.println("Waiting for connection...");
      Socket connectionSocket = welcomeSocket.accept();
      System.out.println("  >Socket connected");

      BufferedReader inFromClient = new BufferedReader(new InputStreamReader(connectionSocket.getInputStream()));
      DataOutputStream outToClient = new DataOutputStream(connectionSocket.getOutputStream());
      clientSentence = inFromClient.readLine();

      System.out.println("  >Received: " + clientSentence);
      capitalizedSentence = clientSentence.toUpperCase() + '\n';
      outToClient.writeBytes(capitalizedSentence);
      System.out.println("  >Replied: " + capitalizedSentence);
    }
  }
}

Pretty simple? We’ve ignored error checking in this code, so it’ll be fairly easy to crash the server. But for a quick demo, it’ll do. The server reads a sentence from the client, and capitalizes the alphabetic characters. It then returns this uppercase version of the sentence. Since we send “Testing…” to the server, we expect the result to be “TESTING…”.

To compile the server code, choose “Tools->External Tools->Compile Java” (or Ctrl+1).

Compiling Java code in TextPad

If your code contained no errors (and it shouldn’t if you’re copying and pasting) then you’ll see a message in the output window that reads “Tool completed successfully”. Now, choose “Tools->External Tools->Run Java Application”. At this point, a command window will appear bearing the message “Waiting for connection…”

If you see this text, then your server has been successfully bound to port 7689.

This means that our server is ready for the game client to connect to it. Run the game, then talk to the NPC. You’ll see a popup dialog with the response from the server.

Our RPG has successfully communicated with the server, sending it a message and receiving one back in return.

The server window will also have a few more messages:

Our server log proves that the communication was real, and also that Java and ruby can "speak the same language" using sockets.

Note that, since we closed the connection, the server is now waiting for another connection to be opened. We could talk to the NPC again, or simply close and re-open the game, then talk to the NPC again. Eventually, we will require each client to hold on to a connection from the server, but for now this proves the concept: networked RPGs in RPG Maker VX should work just fine. Press Ctrl+C to close the window; you might then have to type “Y” and press Enter to confirm that you want the server to halt.

Step 4: Taking a Step Back

At this point, it’s worth pausing for a second to consider what we’ve actually accomplished. There’s a lot of “glue code” (like reading lines of text from ruby) that hides what’s actually going on:

• We’ve created a server that can “receive” connections on port 7689, and a client that can request a “connection” on this port.
• Once connected, a client and a server can send and receive a sequence of characters.

That’s all there is to it. Adding multiple clients, some kind of synchronization, and partitioning our data into “messages” or “game events” are just additional details on top of what the tcp client actually is. For example, instead of sending strings of messages, we could send pairs of NPC x/y co-ordinates. In that way, we could treat “heroes” in separate RPGs as “npcs” in your RPG world. We could send “tiles”, and allow users to build their own houses (as maps) which they can allow their friends to visit. We can send custom avatar pictures for each player, or even co-ordinate some kind of battle system. All of these things can be accomplished simply by sending letters and numbers over a TCP connection.

Step 5: A Slightly More Impressive Proof-of-concept

Our “borrowed” Ruby source code is about 500 lines in total. This is only about 8% of the total RMX-OS code, which includes things like Scenes for connecting to the network and a password system to help protect users’ passwords and privacy. My point is, making anything interesting is going to be a huge undertaking. On the other hand, a simple message box usually isn’t enough to prove to your boss that you’ve accomplished a huge feat like networking a game engine that doesn’t normally expect to be multiplayer. What, then, might a nice, simple prototype look like?

• We don’t want to play around with Scene overloading. So let’s make an NPC that basically allows us to connect to a network, and manage our network configurations.
• When the game begins, the player can choose one of four starting heroes. This information is transmitted over the network.
• At this point, we should create NPCs for each of the other users online. These will just wander randomly for now; they won’t reflect the actual position of the other users.
• The hero can talk to these NPCs to receive “messages” that they have sent (like a chat). Only the most recent message is shown.
• When an NPC gets a new message to display, we can show a “balloon” animation above him –an exclamation mark or something similar.
• To send a message, the hero can talk to another NPC (like the connection NPC). We might use the hero name dialog, or we might capture key presses. In the worst case, we’ll just let the user select some pre-defined messages from a list.

There’s a lot of bullets (e.g., a lot of things to show off) but individually, each is not terribly difficult to accomplish. Let’s get started!

VERY IMPORTANT NOTE: Whenever you change some of your game’s code (scripts), you should re-test using a new game. If not, you risk loading old code from an old save file (it happened once to me, while testing this blog entry) and it is frustrating to track down an error like this. In my case, the initialize method for my Game_Map class was not re-called upon loading an existing save file, and I couldn’t figure out why some class variables didn’t exist anymore.

Game Prototype 1: Autosave and an IP NPC

Our first prototype will have no networking capability at all. We’ll need two maps; one for the player to select an Avatar upon starting a “New Game”, and the other for normal play. Create these two maps and call them “Player House” and “Character Select”. Make them both small (20 x 15 or so) and go to the Character Select map. Give it a fancy layout and set the hero’s starting position to be somewhere on the map.

Our spartan starting map.

Now, add a few NPCs (five would be good), each with the graphics of the first five heroes in the party. If you don’t like the default hero pictures, you should go into the database editor (F9) and change them. Just make sure they match the pictures of the NPCs you just added. While you’re in the database editor, you might want to change the Initial Party (in the System tab) to include only hero 6, and change hero 6‘s walkabout to be something more mysterious. Name him SELECT to avoid confusion.

Five possible Avatars on our starting map.

If that was all too much for you, you might consider reading through a few RPG Maker tutorials –this blog post would likely double in size if I tailored it to complete beginners. (If someone’s willing to do a voiceover for me, I’ll put up a video walkthrough on Youtube.)

Open up the left-most NPC in the Event Editor (double-click) and add the following on page 1:

Talking to an NPC activates this event, which asks the user if she wants this avatar.

To summarize that code briefly:

• Show a text box asking if the player wants this hero.
• Have a “choice” command (Yes/No). We only care if the user picks “Yes”
• If so, set variable 1 (which we call Avatar ID) to be 1.
• We then add the first hero (“Ralph”) to the party.
• We remove the SELECT hero.
• We transport to a random point on the Player House map.
• We then “auto-save” using the following script:

Scene_File.auto_save($game_variables[2])

Now, before we talk about auto-saving, copy and paste the event code from NPC 1 to each of the other 4 NPCs. When you paste the events, make sure to change the text in red to reflect avatar 2, 3, etc. Each player should introduce him or herself, ask the player to choose him/her, and then save his/her details into variable 1 and create a party with only him/her in it. Isn’t gender neutral language fun?

Let’s talk about the auto-save. You can probably guess that “auto-save” is (or will be) a static method in the Scene_File class, and that it will take a save slot ID (which we will store in variable 2. Open the script editor (F11) and browse to Scene_File. Add the following directly underneath Scene File < Scene Base:

  #Save to a slot directly.
  #Indexed from 1, not 0.
  def self.auto_save(file_index)
    fileName = "Save#{file_index}.rvdata"
    file = File.open(fileName, "wb")
    Scene_File.write_save_data(file)
    file.close
  end

We copied most of this code from the do_save method, making sure to construct the right file name. By naming our code as self.auto_save, we make the method static –the user code doesn’t need a (new) object to use the function. Unfortunately, this requires that  write_save_data also be static. Find the following line in Scene_File:

 def write_save_data(file)

…and change it to

 def self.write_save_data(file)

However, this will break normal saving, which we probably don’t want. Find the following function in Scene_File:

 def do_save
   file = File.open(@savefile_windows[@index].filename, "wb")
   write_save_data(file)
   file.close
   return_scene
 end

…and change it to:

 def do_save
   file = File.open(@savefile_windows[@index].filename, "wb")
   Scene_File.write_save_data(file)
   file.close
   return_scene
 end

This way, do_save will still work properly (by calling the now-static function). There are other ways to accomplish this, but this is the easiest to describe.

One tiny problem remains: how do we save the preferred autosave slot into variable 2? I see three possibilities:

• Just pick a slot (e.g., “the first save file”) and use that. This is too hot –it forces an arbitrary choice on the player.
• Give the player a choice box with options “1,2,3,4”. This is too cold –it’s a bit impersonal for players used to the nice save dialog of RPG Maker.
• Force the player to save the game once, and use that value. This is just right –it’s natural to the player, and shouldn’t be too hard to figure out from our end.

Needless to say, we’re doing option 3. Create a new event with no graphic in the top-left corner of the map and give it two pages. The second page should have the “Switch” box checked under “Conditions”, and have “Switch 1” selected. (We call this switch Chose Save Slot.) The first page should have its trigger set to “Autorun”, and have the following event code:

The code for our auto-save enforcing NPC.

In other words, we first tell the user to select an auto-save slot. Then, we show them the save menu. Now we need to check if the user canceled the menu. We do this by setting variable 3 (called TMP Save Count) to be equal to “Save Count” (listed under “Other” in the variables dialog). We then do a simple conditional branch: if this variable is greater than zero, we continue. The first thing we now do is execute a nifty script:

  #Set var[2] to the save slot
  slot = $game_temp.last_file_index + 1
  $game_variables[2] = slot

The variable last_file_index is already part of the game engine. It stores the id of the last slot saved to, starting from zero. (Since save slots are actually numbered from 1, we increment this value.) The default value for last_file_index is zero, by the way, so we can’t just check this to determine if the user canceled or saved.

Moving on, we then set switch 1 to ON. Next, we set the names of heroes 7 and 8 to 127.0.0.1:7689. This is not very important right now; just add the lines and we’ll get back to them later. Finally, we perform our “classic auto-save”

  #Save the game to slow (ID-1)
  Scene_File.auto_save($game_variables[2])

You should memorize this code fragment; we’ll be using it a lot.

At this point, you can run the game, and you will be prompted to save. Following that, you can choose a hero and then walk around the Player House map. Not too bad for a quick-and-dirty auto-save, eh? In fact, if you play around with quitting and reloading games, you’ll find that our code is actually pretty robust; cancel or close the program whenever you want and everything should still work fine when you reload it.

Our auto-save dialog performs impressively.

Go to the Player House map and create a nice, homely setting. Add some ambient NPCs if you want; just make sure there’s a scholarly-looking NPC somewhere near the middle of the map. We’ll use him for connecting to the server. Here’s just one possible setup:

Only the primary NPC matters, but feel free to make this map as picturesque as you like.

This NPC will have a whole lot of functionality wrapped up inside his persona. To make the code more manageable, we’ll put different functions into different common events. The core code for the map NPC looks like this:

The main code of our "Connect" NPC.

To summarize:

• This NPC can either introduce himself or call the “Connect to a server” Common Event, which has the ID of 1. The rest is just fluff.

Open the database editor (F9) and browse to the Actors tab. Name actor 7 SAVED_SERVER_1 and actor 8 SAVED_SERVER_2. Then browse to the Common Events tab. Name common event 1 “Connect to a server” menu. Name common event 2 Prompt for server address. Name common event 3 Prompt to Save Server. Now, for common event 1, add the following code:

CE 1 handles connecting to a server. It is currently incomplete.

This event is quite simple:

• First, it asks the user to choose a server. The first two options are the names of the SERVER characters that we just created. At present, nothing happens when these are selected. The third option allows us to add a new server. It calls the following two common events in sucession.

The second common event is also quite simple, if a bit repetitive:

CE 2 allows us to construct new IP addresses. It's just a little cumbersome.

It performs the following routine five times:

• Tell the user the name of hero SELECTION, then ask for a number. Store this number in variable 4 (TMP server part)
• Append this number to the name of the SELECTION hero using the following script:

    #Set actor's name
    actor = $game_actors[6]
    actor.name = "#{$game_variables[4]}."

  • …note, of course, that the “.” changes to a “:” for entry 4, and “” (nothing) for entry 5.

The third common event simply checks if this is what the user wants:

CE 3 just confirms the user's entry, saves it, and resets the SELECTION npc's name.

It saves, e.g., actor 7 by using the following script:

 #Finalize actor's name
 $game_actors[7].name = $game_actors[6].name

Of course, we reset the SELECT hero’s name to avoid confusion later. We then wrap it all up with an auto-save. And that’s it! We now have a simple way to enter server addresses without requiring any fancy interface or custom save files. Here’s a screenshot of our system in action:

The orange text makes it look at least ten times more professional.

Once the selection has been saved, it shows up on our options list:

The beauty of using an Actor's name to save the IP address is that you can use it in a Message Box without any fancy tricks.

It’s nearly time to move on to our second prototype (which will introduce network elements). Before we get too far ahead, I’d like to highlight some of the merits of our approach:

• We relied for the most part on local scripts, requiring only a single change to the game’s source code. In our defense, the function write_save_data should have been static anyway, and could have been fudged if we were so inclined to create a dummy object.
• By using several Common Events, we avoided the RPG Maker-exclusive temptation to create one messy NPC with all our code. In fact, nearly all events had single-screen pages of Event Commands.
• We used “special heroes” to store our IP addresses. This allowed us to stick with event commands like “Show Choice” instead of requiring custom Scripts for everything.
• By and large, we used mostly RPG Maker constructs, relying on scripts only for tiny globs of “glue code” where the interpreter fell short. Most intermediate RMVX users could probably figure out what our code does, and how to improve it.

Enough resting on our laurels; it’s time to notch up the interest level of our audience.

Game Prototype 2: An Online Community

We strive for incremental improvements, so let’s try adding only the most basic of network functionality:

• When a user Connects, we’ll add an (arbitrary) NPC to the player’s home map.
• When a user Disconnects or closes the game, we’ll remove this NPC.

This will require a lot more foresight than you might expect. First, we need to decide how the client should handle messages. After all, we don’t want our game engine to hang just because the network goes down. Following that, we’ll need a simple message format that is simple to expand with new data types. Finally, we’ll need a messaging protocol: how often should we send updates to users? For now, we’ll just assume “always send every update”, since we’re not tracking anything expensive like a user’s X/Y co-ordinates. For bigger games, the messaging protocol is the most important part of the network design. If your game’s lagging, try reading a few papers on interest management (PDF link) in online games.

First, let’s talk about threading. Like most modern high-level languages, Ruby runs inside a virtual machine, which itself is a single process in the Windows operating system. All ruby-related code in your game must therefore run in sequence. This includes all parallel or autostart events, common events, NPC interactions, and scripts. Unfortunately, this makes it very difficult to add networking. Let’s say that you check for messages from the server before you service all parallel process events. If your game has a lot of these events running at once, the network buffer might overflow, and you’ll lose messages. (Very Bad.) What we really want is a way of telling Ruby to check for network messages “every so often”. That’s exactly what threads were designed to do.

Consider the following Ruby code:

  #Code our thread will execute
  def network_thread
    while true
      read_server_messages()
      send_waiting_messages_to_server()
      sleep(1) #Avoid spinning too fast
    end
  end

  #Actually make a thread
  t1 = Thread.new{network_thread()}

Conceptually, the line with Thread.new tells Ruby’s virtual machine to make the network_thread() function run once inside a new thread. This thread will alternate with the main (RPG Maker) thread in the virtual machine. According to Ruby’s time-slicing policy, each thread will run for 10ms before the other thread gets a turn to run. Inside network_thread, we call a function which reads all messages that the server has sent to us. Note that we do not process these messages yet; we’ll do that in the main game thread. Instead, we simply get them out of the network buffer and into the game engine. Following that, we send any waiting messages to the server. Again, these messages are generated by the game engine; we don’t really care how they’re made. Then, we perform a call to “sleep”, which forces the current thread to suspend. That way, if we only used 2ms of our time slice, the main game engine doesn’t have to wait 8ms while we do nothing useful.

This is only one way of doing things. We might also have put read_server_messages and send_waiting_messages_to_server into their own separate threads.

Now, let’s talk synchronization. Where does read_server_messages save the messages it reads? Presumably, we’ll have an array to store them:

 read_messages = []

The network thread can then put messages into the back of this array, and the game engine can read messages from the front. The problem is, what happens if both try to act on the array at the same time? The answer: something Very Bad. To avoid this, we should synchronize access to this array. For example, the game engine might contain code like the following:

  GameMessage nextMessage = nil
  @lock.synchronize { 
    nextMessage = read_messages.shift unless read_messages.empty?
  }
  if nextMessage
    #....process nextMessage
  end

The @lock variable might be created with a call to:

  @lock = Mutex.new

It ensures that all the code inside the locked block executes before any other locked code. To avoid unnecessary slowdown, it is a good idea to keep the code inside a lock block as simple as possible –note that we store the next message in nextMessage, rather than processing it inside the locked section.

There’s just one problem: Mutex.new requires the ‘thread’ library. And I mean requires; that is, we don’t have access to it in RPG Maker code. (Thankfully, we do have access to thread). We can’t use the equivalent “monitor” class; it’s also required. Fortunately, there’s an easier solution. Remember the ruby 1.8 source code we downloaded earlier? Go to that directory, and then browse to the lib directory. You’ll see the thread.rb file which contains mutexes. Copy and paste the source code into your Ruby Library Code module (F11) at the very top. Segment 2 contains this code, minus most comments:

Code Segment 2 – Ruby code.

And that’s all we need to do to enable mutex creation:

The Mutex object was created properly. (Sample source code not included.)

We nearly hit a dead-end, but synchronization works. Bullet dodged; that could have killed the entire project. (Although not entirely: RMX-OS doesn’t use threads and achieves reasonable performance.)

All drama aside, you should now be able to envision how we can combine synchronization with threading to read messages from the server. But what is a game message? We want something easy to extend, and something that (preferably) doesn’t rely on newlines to specify message boundaries. Typically, we would use an array of bytes. However, the Ruby 1.8 library code we borrowed doesn’t support this, forcing us to read and write strings. There are some very good reasons why we don’t want to force byte arrays into strings:

• For some reason, I can’t get the Socket.read command to work. This would have allowed us to read X bytes from the socket, instead of recv‘s “up to X” bytes. (I had actually written half this guide based on read until I realized it didn’t work. Silly me.)
• If we use recv to read bytes, the default for “un-read” bytes is 0. But what if we actually wanted the number 0? We can get around this (add 1 to each byte sent, or initialize our buffer array with -1 instead of 0) but these “solutions” feel more like hacks –and although I derive great enjoyment from puzzling out another programmer’s creative solutions, I dislike writing obfuscated code myself.

So, we’ll make this simple: a message is just a string, of the format:

• TYPE:xxx:yyy:zzz:\n

In other words, a message is composed of a series of strings, separated by colons, with a newline designating the end of the string. The type of the message determines how many remaining segments are expected, and what they mean. Here are our message types, and their corresponding data.

The first type is CONNECT_REQUEST, sent when a client wants to log in to the server.

CONNECT_REQUEST	Description
Type	The string CREQ
Avatar ID	The ID of the avatar our player has chosen.

The second type is CONNECT_RESPONSE, sent back from the server whenever a client tries to log in.

CONNECT_RESPONSE	Description
Type	The string CRES
Response	If 0, the server refused to allow you to connect. Any other number represents your ID on the server. Clients should save this value.

The third type is USER_CONNECT, sent whenever another user has connected to the server to all other players currently online. A series of these messages is also sent from the server to a client that has just logged on, lest he think he logged in to an empty world.

USER_CONNECT	Description
Type	The string UCON
Player ID	The ID of the player who has connected.
Avatar ID	The ID of the avatar this player has chosen.

The fourth message type is USER_DISCONNECT, sent when a user has disconnected.

USER_DISCONNECT	Description
Type	The string UDIS
Player ID	The ID of the player who has disconnected.

The server manages all user connections, so things like timeouts, etc. are not the client’s responsibility. In some cases, the server may shut down; if so, it might send USER_DISCONNECT messages to all clients. Thus, if a client receives a USER_DISCONNECT message with its own user ID, it can assume that the server will send no further messages. Moreover, if the client sends a USER_DISCONNECT message to the server, it is probably trying to request a disconnection.

Let’s actually get some of this code working. First, we’ll need a Game_Shared class and its respective $game_shared singleton for containing all our shared objects (which require mutually exclusive access). Add the Game_Shared module below Game_Interpreter:

#==============================================================================
# ** Game_Shared
#------------------------------------------------------------------------------
#  Contains all objects that may be shared among threads, and contains mutexes
#  for controllling these accesses.
#==============================================================================

class Game_Shared
  
  #--------------------------------------------------------------------------
  # * Object Initialization
  #--------------------------------------------------------------------------
  def initialize
    @read_messages = []     # Just read from server
    @read_lock = Mutex.new  # Lock for read_messages
    @messages_to_send = []  # To send to server
    @write_lock = Mutex.new # Lock for messages_to_send
  end
  
  #--------------------------------------------------------------------------
  # * Public Accessor Functions --From Server
  #--------------------------------------------------------------------------
  def getNextServerMessage
    ret = nil
    @read_lock.synchronize { 
      ret = @read_messages.shift unless @read_messages.empty?
    }
    ret
  end

  def queueServerMessage(msg)
    @read_lock.synchronize { 
      @read_messages.push(msg)
    }
  end

  #--------------------------------------------------------------------------
  # * Public Accessor Functions --From Client
  #--------------------------------------------------------------------------
  def queueClientMessage(msg)
    @write_lock.synchronize { 
      @messages_to_send.push(msg)
    }
  end

  def getNextClientMessage
    ret = nil
    @write_lock.synchronize { 
      ret = @messages_to_send.shift unless @messages_to_send.empty?
    }
    ret
  end

end

To create the singleton, go into Scene_Title and find the function create_game_objects. Add the following text, right at the end of the function:

  $game_shared        = Game_Shared.new

Now, we will have access to our singleton object for most of the game. The shared object singleton allows us to read and write messages to and from the server, using arrays to queue the messages properly. It handles all locking internally, so we don’t need to worry about Mutexes outside of our Game_Shared class.

We’ll also need a Game_NetMessage class; add it directly below Game_Shared.

#==============================================================================
# ** Game_NetMessage
#------------------------------------------------------------------------------
#  A light-weight class used to store messages sent to/from the server.
#  Contains helper methods to construct most message types.
#==============================================================================
class Game_NetMessage
  
  #Message types
  def self.CONNECT_REQUEST  
    "CREQ" 
  end
  def self.CONNECT_RESPONSE 
    "CRES" 
  end
  def self.USER_CONNECT     
    "UCON" 
  end
  def self.USER_DISCONNECT  
    "UDIS" 
  end
   
  #Helper methods
  def self.Make_ConnectRequest(avatarID)
    "#{Game_NetMessage.CONNECT_REQUEST}:#{avatarID}\n"
  end
  def self.Make_ConnectResponse(playerNewID)
    "#{Game_NetMessage.CONNECT_RESPONSE}:#{playerNewID}\n"
  end
  def self.Make_UserConnect(playerID, avatarID)
    "#{Game_NetMessage.USER_CONNECT}:#{playerNID}:#{avatarID}\n"
  end
  def self.Make_UserDisconnect(playerID)
    "#{Game_NetMessage.USER_DISCONNECT}:#{playerID}\n"
  end  
 end

This class functions entirely through static method calls; you’d never need to create a Game_NetMessage object. Instead, you’d use it like this:

  msgToSend = Game_NetMessage.Make_ConnectRequest(3)
  $game_shared.queueClientMessage(msgToSend)

…to make a connection request message and queue it for sending to the server.

All we need now is a $game_network class which performs the following tasks:

• Create a network thread when the user connects to the server. Destroy this thread (safely) when the user disconnects, or when the server kicks him off.
• Maintain some statistics relating to the network: Is it connected? What’s the latency?
• Make sure that the network thread calls $game_shared.queueServerMessage() when it receives a message from the server. Make sure it sends data to the server whenever $game_shared.getNextClientMessage() returns something non-nil.
• Catch most bad behavior (like trying to connect to a server multiple times, etc.)

Given this brief description, we might create Game_Network (placed directly under Game_NetMessage) as follows:

#==============================================================================
# ** Game_Network
#------------------------------------------------------------------------------
#  This singleton class controls network access
#==============================================================================
class Game_Network
  
  #--------------------------------------------------------------------------
  # * Object Initialization
  #--------------------------------------------------------------------------
  def initialize
    @s = nil                # Our socket
    @loc = ""               # Help with debugging a bit...
    @network_thread = nil   # Thread is non-nil when we have a server connection
    @kill_thread = false    # Set to "true" to gracefully terminate this thread
    @server_message_buffer = [] # Buffer of bytes received from the server
    @my_id = -1 #Managed here, not in the game manager
    @connect_mutex = Mutex.new
    @discon_mutex = Mutex.new
    
    #Always start OFF
    $game_switches[2] = false
  end
  
  #--------------------------------------------------------------------------
  # * Main Functionality
  #--------------------------------------------------------------------------
  def handle_messages
    #Write while we have messages to send
    @loc = "handle_messages():send"
    while true #Can always send
      #Anything to send?
      @loc = "handle_messages():check to send"
      clientMsg = $game_shared.getNextClientMessage
      break unless clientMsg
      
      #Send it
      @loc = "handle_messages():send messages"
      @s.send(clientMsg)
    end
    
    #Read while we have data available
    @loc = "handle_messages():receive"
    partial_message = nil
    while @s and @s.ready? #Can only recv if the socket is ready
      #Read as many remaining bytes as possible
      #NOTE: Using -1 means that there will always be a '' entry if 
      #      our string originally ended in \n
      readStr = @s.recv(0x1024)
      readStr.gsub(0.chr, '')
      
      #TEMP
      #print "Read: #{readStr}" unless readStr.empty?
      
      
      #Continue?
      next if readStr.empty?
      lines = readStr.split("\n", -1)
      
      #Any partial message to complete?
      if partial_message
        lines[0] = partial_message + lines[0]
        partial_message = nil
      end
      
      #Is the final message partial? If so, defer processing till later
      partial_message = lines.pop 
      partial_message = nil if partial_message.empty?
      
      #Ok, we've got a series of messages. Queue them for the client
      lines.each{|message| 
        @loc = "handle_messages():process single message"
        
        #Split this one too; remove newline
        segments = message.rstrip.split(/:/)
      
        #TEMP: DEBUG
        #print "Message: #{message}"
        
        #We don't always post it
        if segments[0]==Game_NetMessage.CONNECT_RESPONSE
          id = segments[1].to_i
          @my_id = id if id > 0
        elsif segments[0]==Game_NetMessage.USER_DISCONNECT and segments[1].to_i==@my_id
          destroy_thread()
        else
          @loc = "handle_messages():try to queue"
          $game_shared.queueServerMessage(message)
        end
      }
    end
  end
    
  
  #--------------------------------------------------------------------------
  # * Thread access
  #--------------------------------------------------------------------------
  def connect(serveraddr)
    @connect_mutex.synchronize { #Keep re-entrant calls legal.
      #Error?
      raise "Network error: already connected" if @network_thread
    
      #Parse out the port from the server address
      serverparts = serveraddr.split(':')
      if serverparts.length != 2
        raise "Network error: cannot connect to \"#{serveraddr}\", bad address"
        return
      end
      serverparts[1] = serverparts[1].to_i
      
      #Create a network thread to connect and parse messages
      # Note that threads must handle their own exceptions
      @kill_thread = false
      @network_thread = Thread.new {
        begin
          #Connect
          @loc = "connect():init connect"
          @s = TCPSocket.new(serverparts[0], serverparts[1])
          $game_switches[2] = true
          
          #Send first message: whoami?
          @loc = "connect():send first message"
          avatarID = $game_variables[1]
          msg = Game_NetMessage.Make_ConnectRequest(avatarID)
          $game_shared.queueClientMessage(msg)
          
          #Process
          while true
            #Done?
            @loc = "connect():check if done"
            @discon_mutex.synchronize {
              break if @kill_thread
            }
            
            #Process any messages
            @loc = "connect():call handle_messages()"
            handle_messages()
            @loc = "connect():returned from handle_messages()"
            sleep(1)
          end
        rescue StandardError => err
          #Print an error, destroy this thread
          print "Exception in network thread(#{@loc}): #{err}\n"
          @network_thread = nil
        end
      }
    }
  end
  
  def ask_to_disconnect
    if @my_id != -1
      #Queue a closing message
      msg = Game_NetMessage.Make_UserDisconnect(@my_id)
      $game_shared.queueClientMessage(msg)
    else
      #Just kill the thread
      destroy_thread()
    end
  end
  
  def destroy_thread
    #Signal the thread to stop
    @discon_mutex.synchronize {
      @kill_thread = true
    }
    if @network_thread and @network_thread != Thread.current
      @network_thread.join
    end
    @s.close() if @s
    
    #Rest to nil
    $game_switches[2] = false
    @network_thread = nil
    @s = nil
    @my_id = -1
  end
  
end

Ok, that was far longer than I’d hoped, but that’s probably because my understanding of Ruby is pretty lousy. Before we analyze it, add the following to the end of Scene_Title‘s create_game_objects function:

  $game_network       = Game_Network.new

This creates our singleton object; now, let’s talk about the main code. First, some disclaimers: even though I’ve added a few mutexes, this code definitely isn’t thread-safe. If you call connect() and destroy_thread() multiple times and quickly, the code will likely destabilize. However, it probably won’t mess up under normal usage conditions. There’s only three  main differences between this code and the snippet we used in Step 3 to test our network connection:

• We borrowed the message loop from RMX-OS’s listen method, although we changed a few things to suit our purposes.
• Although the game engine handles most messages, our Game_Network class automates connection/disconnection requests and the storage of our own ID. We only have to call the connect and ask_to_disconnect methods (and read all other relevant messages in the game thread, of course) for communication to work.
• We use Switch 2, which you should now name Network ON, to store the status of the network. This makes it easy for NPCs to react to “being online” without requiring them to use any scripting.

The client-server communication is only halfway complete; we also have to create client messages to send and respond to server messages. This will occur inside the game engine. Briefly, the following needs to happen:

• Call connect and ask_to_disconnect when appropriate.
• Handle USER_CONNECT and USER_DISCONNECT messages (and discard or err on unknown messages) at some central point in the client code.

That’s it. Our client sends no outgoing messages that aren’t handled in Game_Network (for now). The question remains: where should we check messages? It should be somewhere central, and somewhere accessed often (but not too often). I recommend Game_Map´s update method. Add the following line to Game_Map´s update method, after update_scroll and before update_events:

  update_network

Now, somewhere in that module, add the code for update_network:

  #--------------------------------------------------------------------------
  # * Update Network: Read messages  (write as-needed elsewhere)
  #--------------------------------------------------------------------------
  def update_network
    #"Process no more than X messages"
    nextMsg = nil
    for i in (1..10)
      #Read the next message
      nextMsg = $game_shared.getNextServerMessage
      break unless nextMsg
      
      #Split
      segments = nextMsg.rstrip.split(/:/)
      
      #Process
      if segments[0]==Game_NetMessage.CONNECT_REQUEST
        next
      elsif segments[0]==Game_NetMessage.CONNECT_RESPONSE
          #Always refresh
          $game_map.need_refresh = true
      elsif segments[0]==Game_NetMessage.USER_CONNECT
        #Add an NPC to the map with the right picture and behavior.
        #In range (3,9) , (11,14)
        avtID = segments[1].to_i - 1
        plrID = segments[2].to_i
        if avtID < @max_network_npcs
          #Get the event, place it
          #print "Moving NPC #{@net_npc_start + avtID}"
          newNPC = @events[@net_npc_start + avtID]
          moveNPCRandom(newNPC, 3, 9, 8, 5)
          
          #Set its variable flag (for picture)
          $game_variables[@net_npc_picvar_start + avtID] = plrID
          
          #Always refresh
          $game_map.need_refresh = true
        end
      elsif segments[0]==Game_NetMessage.USER_DISCONNECT
        #Remove the NPC
        avtID = segments[1].to_i - 1
        @events[@net_npc_start + avtID].moveto(1, 1)
        
        #Remove its picture
        $game_variables[@net_npc_picvar_start + avtID] = 0
        
        #Always refresh
        $game_map.need_refresh = true
      end  
    end
  end

This code is not complicated, but let’s review it anyway.

• First, we read through all available messages. Since a very busy server will probably never stop spawning messages, we limit our processing to 10 messages per game “tick” (which should equate to 600 messages processed per second). We might have to change this later.
• Next, we react to each message:
  • When a user connects, we take a pre-defined npc and move it to a random location between (3,9) and (11, 14). We will create placeholder events later. We then set a pre-defined variable to be equal to the avatar’s ID. Presumably, our pre-defined event will have a page for each avatarID with its respective walkabout.
  • When a user disconnects, we teleport that NPC to the top-left corner of the map, and set its avatar ID variable to 0, thus making it invisible.
• A note about $game_map.need_refresh; this function basically forces several computationally-intensive checks of the map to re-occur on the next map update. In particular, we use it to make sure that each event is updated to the proper page when our NPC-Avatar ID variable is set.

This code requires the the function moveNPCRandom; for now, just add this prototype somewhere in the Game_Map class:

  #For now, just non-random is ok
  def moveNPCRandom(npc, xMin, yMin, width, height)
    for y in (yMin..yMin+height)
      for x in (xMin..xMin+width) 
        if events_xy(x, y).empty?
          npc.moveto(x, y)
          return
        end
      end
    end

    #If all else fails, just place it in the middle (there will be a conflict)
    npc.moveto(xMin+width/2, yMin+height/2)
  end

Re-using existing NPCs is not the best way of doing things; unfortunately, the Event class is internal to RPG Maker and cannot (easily) be changed. For our proof-of-concept, however, it will work fine.

Before we talk about the variables @net_npc_start and so on, let’s add the placeholder NPCs to the map. Create five npcs and place them somewhere near the top of the map, like so:

Put our five dummy NPCs out of range. Make sure they have increasing IDs.

You must create these NPCs in order, so that they have increasing IDs. Confirm that they do, indeed, have increasing events IDs (they should be named Event 006, Event 007, etc.). The first NPC should be set up in this way:

• Page 1 has no graphic, and is below the hero. (You might add a graphic for debugging purposes, just so you can confirm when this NPC moves.)
• Page 2 checks if variable 10 (called NPC1-Avatar ID) is 1 or above. It is a random-walk NPC at the same level as the hero, and has the first hero’s walkabout as its graphic. Its event code consists of two lines:
  • Set variable 9 (called CurrTalk NPC) to be equal to 6. (Or, whatever your NPC’s id is).
  • Call Common Event 4 (called Show NPC Message)
• Pages 3 through 6 are just like page 2, except all the red text is incremented by one. In other words, there is a page for each hero graphic.

The first NPC should look something like this, on the last page:

The last page of the first NPC in our series of dummy NPCs.

Now, alter the other NPCs, except increment the blue text by one. You’re basically creating a new NPC for each character in the virtual world, with a variable to change its appearance. The common event Show NPC Message will be used to centralize all communication.

Finally, go back to your Game_Map code, and put the following in the initialize method, directly before the call to create_vehicles:

  @max_network_npcs = 5 #How many max?
  @net_npc_start = 6 #Start at ID 6
  @net_npc_picvar_start = 10 #Start at ID 10

You might have to change @net_npc_start if your first NPC’s ID was something other than 6.

All that’s left is to hook up $game_network.connect() and the disconnect function. Open the database (F9) and go to the Common Events tab. Go to Common Event 1, “Connect to a server” menu, and add the following to the existing “Choice” command.

• For “When \N[7]”, add the script:

    npcName = $game_actors[7].name
    $game_network.connect npcName

• For “When \N[8]”, add the script:

    npcName = $game_actors[8].name
    $game_network.connect npcName

Now, go to the network NPC you created earlier, and add a new page with the same graphic. For this page’s “conditions”, select “Switch 2 (Network ON) is ON”. Then, add the following event commands:

Our "Connect" NPC will ask us to disconnect if already connected.

Described briefly:

• The NPC will ask if you want to disconnect. If you say “Yes”, he will call $game_network.ask_to_disconnect.

At this point, our prototype is complete. The only thing missing is the server. Our server will be written in Java, and it only performs three important functions:

• Track connection requests and responses, as well as disconnections and timeouts.
• Maintain a list of all online users and their avatars.
• Perform required bookkeeping; making sure that users receive all relevant messages.

Here’s the code for this in Java:

import java.io.*;
import java.net.*;
import java.util.*;
public class VXServer implements Runnable {
  //Data
  private int port;
  private Hashtable<Integer, GameConnection> players;

  public VXServer(String port) {
    this.port = Integer.parseInt(port);
    this.players = new Hashtable<Integer, GameConnection>();
  }

  //Get the next available ID
  private int getNextID() {
    for (int nextID=1; ;nextID++) {
      if (!players.containsKey(nextID))
        return nextID;
    }
  }
  //Repeatedly accept new connections
  public void run() {
    try {
      ServerSocket mainSocket = new ServerSocket(port);
      System.out.println("Server started on local port: " + port);
      for ( ; ; ) {
        Socket childSocket = mainSocket.accept();
        int id = getNextID();
        System.out.println("New player connected; given id: " + id);
        GameConnection gc = new GameConnection(childSocket, id);
        synchronized (players) {
          players.put(id, gc);
          gc.startThreads();
        }
      }
    } catch (IOException ex) {
      System.out.println("Main socket error: " + ex.toString());
      ex.printStackTrace();
    }
  }
  public void removeClient(String reason, int id) {
    String msg = "UDIS:" + id;
    GameConnection remPlayer = null;
    synchronized(players) {
      //Remove from hash
      System.out.println("Closing client[" + id + "] - " + reason);
      remPlayer = players.remove(id);
      //Send a message to all other players
      for (int x : players.keySet()) {
        players.get(x).sendMessage(msg);
      }
    }
    //Disconnect
    remPlayer.stopThreads();
  }

  //Main method; processing starts here
  public static void main(String[] args) {
    if (args.length<1)
      new Thread(new VXServer("7689")).start();
    else
      new Thread(new VXServer(args[0])).start();
  }

  //////////////////
  //Helper class
  //////////////////
  class GameConnection {
    //Data
    public int id;
    public int avatarID;
    private Socket s;
    private boolean active;
    private BufferedReader readStream;
    private BufferedWriter writeStream;
    private Thread readFromClient;
    private Thread writeToClient;
    private ArrayList<String> pendingToSend;
    public GameConnection(Socket s, int id) {
      this.s = s;
      this.id = id;
      this.avatarID = avatarID;
      try {
        this.readStream = new BufferedReader(new InputStreamReader(s.getInputStream()));
        this.writeStream = new BufferedWriter(new OutputStreamWriter(s.getOutputStream()));
      } catch (IOException ex) {
        throw new RuntimeException(ex);
      }
      this.pendingToSend = new ArrayList<String>();
      this.readFromClient = new Thread() {
        public void run() { while(active) { readLoop(); } }
      };
      this.writeToClient = new Thread() {
        public void run() { while(active) { writeLoop(); } }
      };
    }
    public void startThreads() {
      this.active = true;
      this.readFromClient.start();
      this.writeToClient.start();
    }
    public void stopThreads() {
      this.active = false;
      try {
        this.readFromClient.join();
        this.writeToClient.join();
      } catch (InterruptedException ex) {}
      try { this.s.close(); } catch (IOException ex) {}
    }
    public void sendMessage(String msg) {
      synchronized(pendingToSend) {
        pendingToSend.add(msg);
      }
    }
    private void writeLoop() {
      //Wait for a message (messages are NON_null terminated)
      String nextMsg = null;
      for ( ;nextMsg==null; ) {
        synchronized(pendingToSend) {
          if (!pendingToSend.isEmpty())
            nextMsg = pendingToSend.remove(0).trim() + "\n";
        }
        if (nextMsg==null) { try { Thread.sleep(10); } catch (InterruptedException ex) {} }
      }
      //TEMP
      System.out.println("Sent message: " + nextMsg.trim());

      //Send message
      try {
        writeStream.write(nextMsg, 0, nextMsg.length());
        writeStream.flush();
      } catch (IOException ex) {
        removeClient(ex.toString(), this.id);
        return;
      }
      //Quit?
      String[] segments = nextMsg.split(":");
      if (segments[0]=="UDIS" && Integer.parseInt(segments[3])==this.id)
        removeClient("Client requested disconnect.", this.id);
    }

    private void readLoop() {
      for ( ; ; ) {
        //Read a message
        String message = "";
        try {
          message = readStream.readLine();
        } catch (IOException ex) {
          //Close the server
          removeClient(ex.toString(), this.id);
          break;
        }
        //TEMP:Debug:
        System.out.println("Read message: " + message.trim());

        //Split
        String[] segments = message.split(":");

        //React to the message
        if (segments[0].equals("CREQ")) { //Connect Request
          //Save Avatar
          this.avatarID = Integer.parseInt(segments[1]);

          //Send this player a connect_response with his ID
          String msg = "CRES:" + this.id;
          this.sendMessage(msg);
          //Send this client's avatar ID to everyone ELSE
          msg = "UCON:" + this.id + ":" + this.avatarID;
          synchronized(players) {
            for (int x : players.keySet()) {
              if (x!=this.id)
                players.get(x).sendMessage(msg);
            }
          }
          //Send to THIS player all previously-connected players.
          synchronized(players) {
            for (int x : players.keySet()) {
              if (x==this.id || players.get(x).avatarID==0) //Might not have been initialized yet.
                continue;
              msg = "UCON:" + players.get(x).id + ":" + players.get(x).avatarID;
              this.sendMessage(msg);
            }
          }
        } else if (segments[0].equals("UDIS")) { //(Request for) User Disconnect
          //Can only disconnect self
          int newid = Integer.parseInt(segments[1]);
          if (newid == this.id) {
            //Send a disconnect message to this player
            // We close the connection after sending our own message
            String msg = "UDIS:" + this.id;
            this.sendMessage(msg);
          }
        }
      }
    }
  }
}

At over 200 lines, the server code is somewhat long, but that’s usually what ends up happening with Java. Our server functions in a relatively simple manner. At first, there is only one thread, containing the ServerSocket. This is basically the “connection” class, which will constantly wait for connections from our clients. Each time a connection is detected, the main thread creates a small helper class with the socket it just opened. This class spawns two more threads, one for reading from the socket and the other for writing to it. Thus, the total number of threads on the server at any one time is equal to 2 X number_of_clients + 1. There are better ways of doing this (e.g., using the NIO library), but they would require us to modify our client (RPG Maker) code in ways that are somewhat difficult.

Anyway, once the server is running, each client thread simply reads and writes messages as one would expect. Because the code for our server is contained within one file, we can access most functions without requiring any Interfacing hacks. All in all, it’s a rather tiny server.

Compile your server (Ctrl+1 in TextPad), and then run it (Ctrl+2). It will use the default port (7689) on localhost (127.0.0.1). Open RPG Maker and run your game. Talk to the Network NPC and select the first connection (assuming you didn’t modify it). Switch back to the server window; hooray! you’re connected.

Our server window has messages from both players.

Run another game, and you’ll see your player there. Disconnect and connect as many clients as you like. However, for now I recommend disconnecting by closing the game client; for some reason, using the NPC to disconnect will occasionally crash the server. (This is undoubtedly a flaw in my server code, not a bug with networking in RPG Maker games.)

Our NPCs connect, and they walk around randomly.

Everything works just fine; when you close the second window, its corresponding NPC will disappear from the first’s window. Isn’t that amazing?

Game Prototype 3 – Chatting with Other Players

We could stop here; our point has been proven. However, Step 2 was particularly time-consuming, and I don’t want you to think that network games are so tough. In fact, with the simple infrastructure we’ve developed for part 2, we can easily add the remaining functionality in a simple, modular way. (Of course, we’ll eventually reach the point where the network becomes saturated with messages. At that point, it will become very difficult to proceed, because we’ll have to start talking about interest management techniques.)

Let’s add client messages as a kind of “poor man’s chat” interface. Surprisingly, the networking is now the easiest part:

CHAT_MESSAGE	Description
Type	The string CMSG
Player ID	The ID of the player sending the message.
Message	The message string.

We will use the same network message to send a message request to the server, and to receive a message from the server (on behalf of another player). Add the following to Game_NetMessage:

  def self.CHAT_MESSAGE
    "CMSG" 
  end

And:

  def self.Make_ChatMessage(playerNID, msg)
    "#{Game_NetMessage.CHAT_MESSAGE}:#{playerNID}:#{msg}\n"
  end

Add the following to Game_Network:

  def sendChatMessage(msg)
    #Note that this MIGHT occur if we haven't heard back from the server 
    # yet. We might have to manually police this case later.
    raise "Cannot send message; you are not connected!" if @my_id == -1
    toSend = Game_NetMessage.Make_ChatMessage(@my_id, msg)
    $game_shared.queueClientMessage(toSend)
  end

Add the relevant message handling code to Game_Map, in the update_network function, right after the code for handling USER_DISCONNECT.

      elsif segments[0]==Game_NetMessage.CHAT_MESSAGE
        #Retrieve the message
        cmsg = segments[2]

        #Save the player's message
        avtID = segments[1].to_i - 1
        @events[@net_npc_start + avtID].chat_message = cmsg

        #Show an emotion icon
        @events[@net_npc_start + avtID].balloon_id = 1
      end  

This, of course, requires a new property to be added to Game_Event. Add the following to the Game_Event class, below the other attr_readers

  attr_accessor :chat_message             # Current message from other user

…and in the initialize method:

  @chat_message = ""

Thus, each NPC now has a property, chat_message, which is updated whenever a network message of type CHAT_MESSAGE. From here, the code for the Show NPC Message Common Event should be obvious. First, hit F9 and go to the Actors tab. Press “Change Maximum…” and enter a number like 15. Then, name NPCs 10, 11, 12, and 13 to CurrNPCTxt Line 1, CurrNPCTxt Line 2, CurrNPCTxt Line 3, and CurrNPCTxt Line 4. Now, edit the Show NPC Message Common Event:

Despite its length; this common event is actually quite simplistic in nature.

This is one of those “exploding events” that PRG Maker famously espouses. Briefly described:

• We first run a simple script that takes the current NPCs chat_message and splits it whenever there’s a tab (\t). (We can’t split on newlines because newlines denote new messages in our network protocol.) The reason our script is so complex is so that we can handle cases such as the empty string (“no message”) or a string with less than 3 tabs (messages with less than 4 lines).

   #What NPC is talking? What's he saying?
   currMsg = $game_map.chat $game_variables[9]
  
   #Set NPCs 10..13 to be the lines of text
   lines = currMsg.split("\t", -1)
   rep = lines.empty? ? 4 : 4-lines.length
   lines += [""]*rep
   for id in (0..3)
    $game_actors[10+id].name = lines[id]
   end

• Next, we run a much smaller script with sets the value of variable 8 (which you can rename to Curr NPC Avatar) to be equal to value of the avatar variable for the given NPC. I thought this kind of functionality was offered by RPG Maker’s standard events, but I couldn’t find it anywhere.

   #Set variable 8 to the current NPC's avatar
   index = $game_variables[9] - 6 + 10
   $game_variables[8] = $game_variables[index]

• Next, we have a big mess of If statements. Each “if” statement wraps the following text box command:

     \N[10]
     \N[11]
     \N[12]
     \N[13]

• …this uses the values we just stored in the Actors’ names to spell our out message. The only difference between each text box in the if statements is their face picture; we change this based on the value of variable 8 (Curr NPC Avatar).

The first script in our event will require the following method to be added to Game_Map:

  #Helper
  def chat(npcID)
    msg = @events[npcID].chat_message
    msg ? msg : ""
  end

The name was kept short to prevent overflowing the tiny script box inside RPG Maker’s Script Event Command. The reason we check for nil values and replace them with the empty string is that some of the functions that call chat will expect strings to be returned.

So now, messages work –presuming that we can actually send one. The problem is, text entry in RPG Maker is notoriously obnoxious. We could use the “Hero Name” dialog to spell out the sentence word-by-word, and then display the partial sentence like we did with the IP dialog. This would be clumsy at best. We could provide a set of partial phrases, and allow the player to select lines 1 though 4. This would actually be a minor pain too. From the user’s point of view it would be simplest to make our own Scene_* class that traps key events and lets them just type out the sentence. But that could easily double the length of our simple prototype. Since the problem cannot be solved simplistically, I’ll just punt on it: we’ll give the player a set of four predefined messages. First, add the following event command to the end of Common Event 4, Show NPC Message:

   Call Common Event 5

Name Common Event 5 something like Respond to NPC and give it the following code:

Event that allows a player to communicate with everyone else.

This Common Event is easy to understand, although it is clearly a too cold type:

• Ask the player if he wants to respond to that NPC.
• If so, let them choose a simple (one-line) message, a multi-line message, or just a glitchy message (to test what happens). I know, boring, huh?
• Then, simply use the function we put in $game_network to send the message.

    msg = "Hi there, friend!"
    $game_network.sendChatMessage(msg)

Pretty simple, although I think it’s a bit of a bad abstraction (since we talk to a single NPC to change our message to all NPCs. But that’s a minor point; we could change this easily later if we wanted to. Anyway, all that’s left is our server code. In the function readLoop, add the following case:

        } else if (segments[0].equals("CMSG")) { //Send a chat message to all clients
          //Can only send messages from yourself
          int fromID = Integer.parseInt(segments[1]);
          if (fromID == this.id) {
            //Send the same message to all other users
            synchronized(players) {
              players.get(fromID).lastMsg = segments[2];
              for (int x : players.keySet()) {
                if (x!=this.id)
                  players.get(x).sendMessage(message);
              }
            }
          }
        }

You’ll have to change the definition of GameConnection to include this “last known message” parameter:

    public String lastMsg;

…and, in the constructor:

      this.lastMsg = "";

You will, of course, need to add a similar broadcast to the UCON case. We can modify the last use of synchronized(players) to include this:

          //Send to THIS player all previously-connected players and their messages
          synchronized(players) {
            for (int x : players.keySet()) {
              if (x==this.id || players.get(x).avatarID==0) //Might not have been initialized yet.
                continue;
              msg = "UCON:" + players.get(x).id + ":" + players.get(x).avatarID;
              this.sendMessage(msg);
              msg = "CMSG:" + players.get(x).id + ":" + players.get(x).lastMsg;
              this.sendMessage(msg);
            }
          }

I’ve named this server VXServer2, and included its full listing in Code Segment 3, below, in case you’re not familiar enough with Java to insert the code yourself.

Code Segment 3 — Make sure this is in a file called VXServer2.java, or it won’t compile.

Anyway, compile it (Ctrl+1) and run it (Ctrl+2) then try out our nifty message code.

It looks like your friend has something to say to you.

The exclamation also appears at potentially inappropriate times, like when the NPC first logs on, but I don’t think that’s such a bad thing –it helps by alerting the player that a new NPC has connected.

A message which, if not user-defined, is at least user-chosen.

The server scales nicely, although if you’re running these examples locally, you’ll be limited by the fact that RPG Maker “pauses” all game windows except the currently-active one (so messages won’t be received). Nonetheless, the effect is impressive:

Several NPCs; we are talking to one while another indicates that he has a new message for us. Pretty cool.

Project

For the first time, I’ve decided to release the final project’s code as a sample. Here’s a zip containing everything you’ll need; make sure you read the article to understand how it works. As always, all code is in the public domain, unless it is borrowed from another author (in which case I have noted it) and is thus released under the license chosen by the respective author. All such licenses are open-source in nature.

At the moment, I cannot find a good site to host the project file. However, I posted the sample code on the RPG Maker VX Forums; you can download it from here (but you’ll need an account). If I can find a good hosting site, I’ll upload a direct link. Apologies for the inconvenience, readers (but it’s a nice forum anyway).

Further Work

I’m personally quite pleased with what we’ve accomplished; however, it’s by no means polished. There’s still bugs in the system, and there’s a great many improvements that could be made. Here’s some “homework” for you; I might get around to writing a post about one of these eventually, but for now I’m tired of writing server code.

• BUGFIX: Send a “glitch” message (option 3) and then try to read it with another player. Surprisingly, the client will crash! This was actually unintentional on my part, but it presents a nice, easy exercise for you. Figure out where the system is glitching up: server side, client side, or somewhere in between. Then fix it!
• BUGFIX: For some reason, closing the network connection from the NPC might crash the server. Also, hitting “F12” to go back to the title screen might have a negative effect on the connection. Re-write parts of the server and client to be more robust at handling this type of thing.
• FEATURE: Once connected, we should measure the average lag to the server, and print this in the lower-right corner of the screen. Perhaps use the Debug Window to design some kind of “connected” box which always hovers in the lower-right corner. Bonus points if you animate it with a spinning Internet icon.
• FEATURE: Combine this article’s source with the NPC Talk Boxes one, and make NPC messages appear above their heads whenever they change. You can still show the exclamation emote when they first connect.
• FEATURE: Our NPC code uses “sir” a lot, even if the connected player is clearly a female. Extend the game engine to handle strings like \sir, which expand into “sir” if the player is male and “ma’am” if the player is female.
• FEATURE: Add user accounts and passwords, and do this in a way that keeps users’ passwords safe and un-crackable –a salted hash comes to mind.
• PROJECT: Allow players to fight each other. This isn’t as difficult as it seems, since RPG Maker’s battle system is turn-based. You can extend Scene_Battle to expect network input and wait until both teams have made their decisions. Then, you might add an icon to show if the other player is “ready”, “disconnected”, etc. Finally, you’ll have to take the opposing player’s “stats” and somehow turn them into monsters which you place on the opponent’s screen.
• PROJECT: Assuming the previous project has been completed, add the ability for players to form “teams” and fight against each other at the party level. You can update your heads-up-display to show the moves your teammates have chosen, and you’ll probably want a team-chat and battle-chat feature (with real-time typing, not RPG Maker’s name entry dialogue). I’m giving this project a gold star, since this is the project I would most like to do myself.
• PROJECT: Extend our code to handle events arbitrarily. In other words, make it so that we can do something like $game_map.add_npc(Event(3, 5)) to create a new NPC at location (3,5). This is harder than it sounds, and really only makes sense when preparing for the next item.
• PROJECT: Track NPC movement in real-time. You might have to move our update_network code to a more prominent location –say, the place where RPG Maker handles user input. Regardless, the key challenge with this project is network congestion; sending 60 messages per NPC per second is a sure way to bog down your server. In other words, only update what’s necessary. You might, for example, send a USER_LEFT message when the player wants to move left. The server can then determine if moving left is allowed, and send back OK_TOGO_LEFT or NOGO_LEFT to inform the player. However, this will introduce some noticeable lag on the client side, so you might want to allow the client to begin moving left anyway, and just teleport him back if NOGO_LEFT was received. A careful consideration of when to attempt a move before receiving a response from the server is difficult, and will require extensive testing. On that note, you’ll have to test this code from multiple computers, since (1) there’s no lag otherwise and (2) RPG Maker pauses all but the top-most window. In fact, this pausing is a problem, since it interrupts the flow of messages. You’ll need to account for that somehow (perhaps force the game to remain full-screen? Or install a Windows hook that tricks RPG Maker games into thinking they’re always a top-level window?). Either way, this project will easily take weeks of your time to get right.

Happy coding!

Update: Phoenix Demo

Speed used the code in this post (along with tons of other cool scripts, including a mouse capture script!) to make a really awesome demo called Phoenix. Check it out here.

Some screenshots:

My character, loaded into the main game world.

Loading of a second character, tracking down the first.

New features added by Speed include:

• Characters are tracked by x/y coordinates, and across various maps.
• Full-featured chatting, account creation, password storage, etc. (He saves data in Text files).

…plus a lot more. Pretty cool to see my code in action. And now, back to listening to OC Remixes.