November 3, 2009

[EL-2] Eternal Lands Tray Notifier – Part 1

Blog Info: I finally did it: I took on a task that was too big for me to finish. Fortunately, I managed to get its basic functionality all done by press time. If you can follow along, you’ll know more than enough to continue on your own from here.

Quick Note: I’m still trying to fix some of the formatting, but the content’s all here, so I decided just to post it. I’ll re-tab the untabbed code later. :)

Back-story: Our previous notifier was pretty hackneyed. It was also heavily coupled with the client program’s source, which might not be such a good idea. (What if the developer released the next version in Haskell or Forth? What if he closed the client source for the next release? What if —and this is the most likely— we don’t want to make our notifier any fancier using Win32 C++?) We need a new solution, and that means it’s time to pull out the big guns.

Goal: Write a new notifier plugin. Make it operate external to the Eternal Lands program. Don’t slow down the client or the server. Don’t break any rules. Allow multiple types of notifications. Make it flashy.

Licensing Notice

This post provides a lot of code. While I normally provide all code in the public domain, I am restricting this post’s code with a license. Scroll to the bottom of the post to read it. This license is not for my benefit, but for the benefit of the EL servers and admins. Basically, it prohibits you from using this code to automate, or to modify it in any way that causes undue strain on the servers. You cannot inject events, nor can you delay sending them between the client and server. Finally, you cannot modify the thread priority of the event handler. The EL team has volunteered their time and creativity for your benefit, and I won’t have my code used to cause trouble for them.

Step 1: General Approach

Most of the problems in our last notifier stemmed from hastiness. In light of that, this section will take it slow and explain why we chose to do things the way we did. We’ll discuss some alternative approaches. But first, a general overview of what we want, and why our last plugin is no longer sufficient:

  • Event Notification — We want to see a message when something happens in Eternal Lands. Our initial program tagged harvest completion, but that was all. (Bonus question: How can you stop harvesting without displaying the “You stopped harvesting” message? It’s a tricky one… if you’re not sure, try harvesting some Seridium until you run out of Matter Essences.)
  • Notification Timeliness — We want to show notifications only when necessary, and hide them when they’re not needed anymore. The previous plugin showed them if the Eternal Lands window was not the foreground window, and hid them when a new message came in. Later, we found this to be unsatisfactory. Better to hide all messages when the EL window gains focus.
  • Variety of Notifications — Seeing a non-descript “You stopped harvesting” is different from seeing a shiny “You found an Enrichment Stone!” or a red “You have died and gone to the underworld”. We’ll add multiple types of notifications, which will necessitate allowing them to stack, fade in, and fade out. If you guessed that we won’t be doing all this graphical work in Win32 C++, you guessed right!
  • Separation from Client Code — We won’t be using C++, nor will we be building directly on top of the Eternal Lands source. Rather, we’ll build a separate module that will load the EL client, and then filter all messages sent to it. We also want a system tray icon, and we might even consider removing the Eternal Lands window from the taskbar entirely.
  • Fast and Extensible — We want the code to be fast (sorry, Perl). We want it to be easy to add new filters (sorry, C). We want it to operate directly on the network data (sorry, Forth) and have a great graphics library. If it’s native to Windows, all the better. Our choice is C#.

Why not just read the client log file? If you browse to “C:\Users\User Name\Documents\Eternal Lands\main\chat_log.txt” —possibly replacing “main” with the name of the server you plan on, and “Documents” with “My Documents” on Windows 7— you’ll find a nice log of (most of) the chat window. Fiddle a bit more, and you’ll find that it’s updated in real time. Why, then, do we not simply tail this file? In Windows, this is pretty easy to do in Java:

public void run() {
  File f = new File("C:\\ ... \\chat_long.txt");
  long fPointer = ;/* Seek to the end of the file */
  for (;;) {
    Thread.sleep(100);
    long len = f.length();
    if (len < fPointer)  //Log was reset
      fPointer = len;
    else if (len > fPointer) { //File was appended to
      RandomAccessFile raf = new RandomAccessFile(f, "r");
      raf.seek(fPointer);
      String line = null;
      while ((line = raf.readLine()) != null)
      processMessage(line);
      fPointer = raf.getFilePointer();
      raf.close();
    }
  }
}

Don’t use this code; it doesn’t check for exceptions, and it isn’t mine (it’s part of the java-tail project). More importantly, it doesn’t really work if the file is being constantly added to. You could re-read the file after exceptions, but then you’re entering into the messy possibility of reading the same line twice.

On a totally different level, this is sub-optimal because the client log doesn’t contain everything. What if you wanted to write a filter that tells you if you’ve been interrupted from harvesting by a PKer instead of a sub-lethal cave-in? Battles don’t send messages to the console, unless you die. Wouldn’t you rather find out while you still have a fighting chance?

Why not use Java? This question applies to any other language, but I feel the most reasonable alternative is Java. (I consider Visual Basic .NET to be identical to C# for all such discussions, by the way.) Java is a very good language, and there is no reason not to use it. Your favorite language is also probably a good choice, although I might not have heard of it. My reason for choosing C# was that, of all the candidates I considered, it was capable of (read: “I knew how to use it to”) solving all the sub-tasks I needed. Feel free to follow along in Python or Ruby or Tcl/Tk, converting the code as you like. Ooh, or Delphi; I like Delphi. Regardless, I’m doing this in C#, and any intermediate programmer should be able to follow along.

Why not patch the client code? My earlier reasons for not simply extending on our earlier project were true, but also somewhat motivation by the fact that I knew I was going to use a much higher-level (and thus incompatible) language from the start. What if you started this project using C++? In that case, I’d still encourage you to keep your code totally separate. Besides all the previously-listed reasons, here’s one more: bugs. If you’re recompiling the entire Eternal Lands code base with each new notification, then you’re bound to slip up from time to time. This is part of development; you can release bug-free software, but your internal builds will be fraught with bugs. Now, given the nature of bugs in C++, this means that your code could easily corrupt some of Entropy’s code. If this results in sending a million messages a second to the server, “ya might get banned”. That said, if your bug is passive, and other people use your plugin, you start to run a real risk of bringing down the server. And even if your bug is harmless, it will still confuse developers trying to nail it down. Trust me when I say that tracking a bug to third-party patches does not make developers happy.

How will we catch messages? We’ll start our program first, which should give it priority over any child processes and their system resources. Then, we’ll connect our program to the Eternal Lands server, and connect the EL Client to our program (by adding a new server to the config file). Finally, as we ferry messages between client and server, we’ll send them off to a separate, low-priority thread where we can match then against an arbitrary number of filters, spawning pop-up windows as we identify messages of interest.

Step 2: Connecting to the client

The first thing we need to do it connect to the server. From the Eternal Lands server config file (D:\Programs\Eternal Lands\servers.lst), we know that the “main” server connects to port 2000 on game.eternal-lands.com. Open up Visual Studio (the free Express Edition for C# is pretty nice) and start a new project. Make it a “Windows Forms Application”. You’ll be presented with the design view for “Form1.cs”. In case you aren’t familiar with Visual Studio, take a minute to note the toolbox (circled in red on the left) and the Solution explorer on the right. Right click on Form1.cs in the solution explorer and note the two options shown circled in blue. “View Designer” will bring up the form designer, like you see now. “View Code” will bring up the code that runs everything behind the scenes. Note that C# supports “partial” classes, so the “View Code” option will not show you any code generated by Visual Studio, unless you are running a very, very old version of Visual Studio.

Visual Studio's "Hiding Tool Bars"

Visual Studio's "Hiding Tool Bars"

Open the toolbox (by clicking on it) and drag a “Text Box” control onto your Form. (This is under the “Common Controls” group). Now, right-click on this box and choose “Properties”. The most important property is called “(Name)”, because this is what we’ll use to identify the control in our C# code. The following table details which controls to add, and what properties to set for them. All other properties can be left at their defaults, or you can change them to your liking. For example, you might change the “font” for your controls, or the “BorderStyle”. Go for it.

TextBox*
(Name) txtPathToELFolder
Text C:\Program Files\Eternal Lands

*Note: If you installed Eternal Lands to a different location, set the Text property accordingly. Escaped backslashes are not necessary when entering this string in the visual editor.

Button
(Name) btnRead
Text Read
ComboBox
(Name) cmbServers
Enabled False
Button
(Name) btnConnect
Text Connect
Enabled False
Label
(Name) lblZero
Text 0
TextAlign MiddleRight
Font Courier New, 14.25pt, style=Bold
ForeColor DarkRed
AutoSize False
Size 346, 399
TextBox
(Name) txtConsole
ReadOnly True
Multiline True
Size 306, 188

Arrange these components like so:

Line Up Your Components Roughly Like This

Line Up Your Components Roughly Like This

A quick note on Hungarian Notation: in general, I do not support this hackneyed way of naming variables, especially in a strongly-typed language like C#. However, for GUI components, I often find myself saying “Ah, what was the name of that button I wanted?” By naming controls in an ordered fashion, I can type btn<Space> and get some help from Visual Studio’s auto-complete feature. Having large numbers of (essentially) global variables for your form controls is unavoidable in C#, and I find this approach scales really well, despite the frown that most people get when they see Hungarian Notation.

Compile your program (Build –> Build Solution) and then run it (Debug –> Start Without Debugging). None of the software logic is hooked up yet, but the general idea is that the user can change the client directory, then click “Read” to read the list of available servers. Next, the user can choose a server to connect to from the combo box, and click “Connect”. Log messages will be shown in our multi-line Text Box control, and the mysterious “Zero Count” will be used for something which will be explained later. Well, let’s start the magic.

Click on the “Read” button, and go to its properties. Notice that little lightning bolt, circled below in red? If you click that, you will switch to “Event” triggers. Click the button left of it to go back to properties view.

Bad Design Puts This Very Important Button in a Very Obscure Place.

Bad Design Puts This Very Important Button in a Very Obscure Place.

Click the event button, find the event called Click, and type “ReadELFiles” in the box. Then, press “Enter”. Visual Studio will bring you to Form1.cs’s Code View, with a skeleton implementation of the ReadELFiles function used for your Click event delegate. Java users will be saying “What?”, as delegates and events make little sense from a minimalist point of view. We’ll have a good example of the power of delegates later.

Here’s the implementation of ReadELFiles:

private void ReadELFiles(object sender, EventArgs e)
{
  //Reload an entirely new list of servers.
  cmbServers.Items.Clear();

  //Don’t load anything if we specified an invalid directory.
  if (!File.Exists(txtPathToELFolder.Text + "\\" + "el.exe"))
  {
    txtConsole.Text = "Error: el.exe doesn't exist in EL folder";
    return;
  }
  else if (!File.Exists(txtPathToELFolder.Text + "\\" + "servers.lst"))
  {
    txtConsole.Text = "Error: servers.lst doesn't exist in EL folder";
    return;
  }

  //Open our servers file and read it.
  bool foundLoopback = false;
  StreamReader srvFile = new StreamReader(txtPathToELFolder.Text + "\\" + "servers.lst");
  while (!srvFile.EndOfStream)
  {
    String line = srvFile.ReadLine();
    if (line == null)
      break;

    //Coment? Empty?
    line = line.Trim();
    if (line.StartsWith("#") || line.Length == 0)
      continue;

    //Read:
    //ID, Config_Dir, Address, Port, Description
    String regex = "([^ \t]+)[ \t]+([^ \t]*)[ \t]+([^ \t]*)[ \t]+([^ \t]*)[ \t]+(.*)";
    Match m = Regex.Match(line, regex);
    ServerInfo sInf = new ServerInfo();
    if (m.Success && m.Groups.Count == 6)
    {
      sInf.id = m.Groups[1].Value;
      if (sInf.id.Equals("loopback"))
        foundLoopback = true;
      sInf.configDir = m.Groups[2].Value;
      sInf.address = m.Groups[3].Value;
      try
      {
        sInf.port = Int32.Parse(m.Groups[4].Value);
      }
      catch (FormatException)
      {
        //Don't add
        continue;
      }
      sInf.description = m.Groups[5].Value;
      cmbServers.Items.Add(sInf);

       //Loopback?
      if (sInf.id.Equals("loopback"))
        loopbackInfo = sInf;
    }
  }
  srvFile.Close();

  //Do we need to add our own local server?
  if (!foundLoopback)
  {
    //Copy existing server file
    StreamReader inFile = new StreamReader(txtPathToELFolder.Text + "\\" + "servers.lst");
    StreamWriter outFile = new StreamWriter(txtPathToELFolder.Text + "\\" + "servers.lst.new");
    for (String line = inFile.ReadLine(); line != null; line = inFile.ReadLine())
      outFile.WriteLine(line);
    inFile.Close();

    //Add our own category
    outFile.WriteLine(srvLine);

    //Close, swap, notify.
    outFile.Close();
    File.Delete(txtPathToELFolder.Text + "\\" + "servers.lst");
    File.Move(txtPathToELFolder.Text + "\\" + "servers.lst.new", txtPathToELFolder.Text + "\\" + "servers.lst");
    ShowConnectMsg("Created loopback entry.");

    //Add a new item to the list
    loopbackInfo = new ServerInfo();
    loopbackInfo.id = "loopback";
    loopbackInfo.configDir = "main";
    loopbackInfo.address = SERVER_ADDR;
    loopbackInfo.port = SERVER_PORT;
    cmbServers.Items.Add(loopbackInfo);
  }

  if (cmbServers.Items.Count > 0)
    cmbServers.SelectedIndex = 0;
}

Make sure you add the following using directives to the top of Form1.cs, otherwise you’ll get errors stating that “The name <something> does not exist in the current context”. You also won’t get auto-complete functionality until the proper directives are added. (Auto-imports is something I miss dreadfully from Eclipse.)

  using System.Text.RegularExpressions;
  using System.IO;

You will also need the following variables; add them as part of the class (e.g., one line after the opening brace following “public partial class Form1 : Form”)

public const string SERVER_ADDR = "127.0.0.1";
public const string SERVER_PORT = 5421;
private string srvLine = "127.0.0.1";"loopback        main            " + SERVER_ADDR
      + "               " + SERVER_PORT + "    Loopback adapter for use with the EL Notifier";
private ServerInfo loopbackInfo;

We also need this “ServerInfo” structure everyone’s been talking about. Add it as part of the namespace (not the class Form1).

public struct ServerInfo
{
  public String id;
  public String configDir;
  public String address;
  public int port;
  public String description;

  public override String ToString()
  {
    return description;
  }
}

The “ToString()” method is called by the Combo Box control to display the Object it contains; overriding it allows us to dictate that ServerInfo items are displayed by the description. There are better (but less concise) ways of controlling Combo Box output.

Note that structs in C# are very different than structs in C++. Indeed, you could have used a class here; the only difference would have been in its copy semantics, and possibly its memory footprint.

The only thing you’re missing is the ShowConnectMsg delegate, but for now just assume it’s a standard function call that does some non-essential logging. Let’s dissect the code we just wrote!

The SERVER_ADDR and SERVER_PORT variables help us to set up our program as a middleware between the EL Client and the EL Server. Our program has both client and server aspects; these two variables define access to the latter. We use srvLine to shim the servers list so that it can access our “Loopback” interface. Note that srvLine is effectively constant, even though it is not declared that way.

We are basically equivocating on the program’s socket expectations. From Wikipedia:

An Internet socket is characterized by a unique combination of the following:

  • Protocol (TCP, UDP or raw IP).
  • Local socket address (Local IP address and port number)
  • Remote socket address (Only for established TCP sockets.)

The original connection from EL Client to EL Server has one socket: a TCP-enabled connection from 127.0.0.1:???? to game.eternal-lands.com:2000. This socket is bi-directional. We don’t know the local port, because local ports are almost always randomly assigned. We don’t technically know the server port either, but 2000 is what we connect to, so I’ll list that here.

Our shimmed middleware creates two bi-directional sockets:

TCP, 127.0.0.1:???? ? SERVER_ADDR:SERVER_PORT
TCP, SERVER_ADDR:???? ? game.eternal-lands.com:2000

Note that, as per the usual, we don’t know the local ports. Either way, we are guaranteed that they will be unique.

All that explanation for just three lines of code? Well, if you’re a non-network guy like me, figuring this out is one of the hardest parts of this project. Now, on to the rest of the code:

The ReadELFiles function is pretty easy to follow:

  • Note that the function parameters “sender” and “e” are not used in our function. These are mandated by the .NET platform, but are not needed for our simple example.
  • The foundLoopback variable is used to check each line read from servers.lst. If the id of that line is “loopback”, then we know a loopback entry exists. If, at the end of scanning, no such entry was found, we just add it to the end of the file. That way, when we start Eternal Lands, we can connect to the loopback server (instead of main or pk or testing) and fool the client into thinking it’s connecting to the real server, instead of our fake, localhost middleware.
  • Note that we should check the loopback entry to ensure that its address and port parameters match ours. I’ll leave this as an exercise in regular expressions.
  • Some regex pattern matching is used to filter each entry in our servers.lst file. We use regexes to check if a line matches a specific pattern, and we then use the Groups property to extract what was matched. If you don’t understand pattern matching, just skip the details. Learn it later, of course —it’s a fantastic way of dealing with text in almost any modern language.

We will now add the final missing piece of the puzzle. Add the following to the namespace (not the Form1 class):

public delegate void ShowConnectMsgDel(String msg);

Then, add the following function definition to the Form1 class:

private void ShowConnectMsg(String msg)
{
  Control ctl = txtConsole;
  if (ctl.InvokeRequired)
    ctl.Invoke(new ShowConnectMsgDel(ShowConnectMsg), msg);
  else
    ctl.Text = msg;
}

Our program will now compile. (I’ll explain delegates later, when it makes more sense to.) Run it, click the “Read” button, and you will see “Main game server” appear in the still-disabled Combo Box.

A Properly-Formatted Servers File Is Parsed.

A Properly-Formatted Servers File Is Parsed.

If your program doesn’t compile, have a look at Listing 1 and try to figure out where you went wrong. (Apologies for the PDF; WordPress doesn’t like .txt files.) If it doesn’t load the server list, make sure you supplied the proper path to the EL Client folder.

Now, add the following code to the end of your ReadELFiles function:

//Enable the remainder of our form
cmbServers.Enabled = true;
btnConnect.Enabled = true;

That should enable everything you need. Now, in Design mode, click on the “Connect” button and give it a click Event called ConnectToServer. You’ll be given a function skeleton again; here’s what we’ll add to it:

private void ConnectToServer(object sender, EventArgs e)
{
  //Connect to our server
  UpdateConsole("Connecting.....");
  Thread t = new Thread(new ThreadStart(Check_Server));  
  t.IsBackground = true;
  t.Start();
}

This requires the CheckServers function, which will run in a separate thread:

private void Check_Server()
{
  //Our middleware acts as a client in this case.  
  TcpClient tcpclnt = new TcpClient();
  Stream srvStream;

  //Get our currently selected server and start it
  ServerInfo sInfo = (ServerInfo)cmbServers.SelectedItem;
  tcpclnt.Connect(sInfo.address, sInfo.port);

  //Display any messages we receive.
  srvStream = tcpclnt.GetStream();
  byte[] msg = new byte[1024];
  bool done = false;
  for (; !done; )
  {
    int amtRead;
    try {
      amtRead = srvStream.Read(msg, 0, msg.Length);
    } catch (IOException) {
      UpdateConsole("\r\nIOException; closing Server");
      amtRead = 0;
    }

    if (amtRead != 0)
    {
      //Echo to console
      String line = "MSG: [";
      for (int i=0; i<amtRead; i++)
      {
        byte b = msg[i];
        if (b >= 32 && b <= 126)
          line += (char)b;
        else
          line += '.';
      }

      line += "]";
      UpdateConsole("\r\n"+line);
    }
    else
    {
      UpdateConsole("\r\nRead zero; closing Server");
      done = true;
    }
  }

  //Must close BOTH
  srvStream.Close();
  tcpclnt.Close();

  //Ready to go again?
  UpdateConsole("\r\nServer Stopped");
}

Make sure you add the following using directives to the top of your file:

using System.Threading;
using System.Net.Sockets;

After that, you’ll only get errors for the UpdateConsole function, which we’ll implement now:

private void UpdateConsole(String msg)
{
  Control ctl = txtConsole;
  if (ctl.InvokeRequired)  
    ctl.Invoke(new AddToConsoleDel(UpdateConsole), msg);
  else
    ctl.Text += msg;
}

Add the following to our namespace, and we’ll be done:

public delegate void AddToConsoleDel(String msg);

Run the program, click “Read” to get a server list, then choose the “Main game server” from your dropdown list and hit “Connect”. You should see a nascent echo of the regular log-in message, and some unreadable garbage.

The "Garbage" Is Actually the Important Information, As We'll See Later.

The "Garbage" Is Actually the Important Information, As We'll See Later.

If this just doesn’t work, have a look at Listing 2 to see where you went wrong.

If you scan the code, you’ll find everything in order. First, we create a TCPClient object to manage our TCP connection to the server. We then Connect it to the server, based on the item chosen in the drop-down list. Finally, we continue to read up to 1kb at a time from the server. To the best of my knowledge, the Eternal Lands server never sends more than a few hundred bytes of data at once, so we don’t have to merge messages.

Note that this all has to happen in a different thread, to avoid throttling the GUI dispatch thread. If you just ran a tight loop in the main Thread, your Windows Form would become sluggish and unresponsive. The C#-ism:

  Thread t = new Thread(new ThreadStart(Function_Name));

…capitalizes on delegates to encapsulate a function (Function_Name) in a new thread. This is a reasonably good example of a delegate, but we have a much better one: our UpdateConsole function.

The problem with ThreadStart is that it’s too easy to guess what it does, without actually understanding why we choose to do it this way. Instead, let’s have a look at UpdateConsole. I’ve highlighted it differently below:

private void UpdateConsole(String msg)
{
Control ctl = txtConsole;
if (ctl.InvokeRequired)
ctl.Invoke(new AddToConsoleDel(UpdateConsole), msg);
else
ctl.Text += msg;
}

First, our variables are declared in red. One is the value we wish to update the console with (as a String) and the other is a local variable used to reference the Text Box itself. Since all Text Boxes subclass Control, we can store this variable in a generic fashion. Note that the variable ctl was created to emphasize that this technique can be used on any part of our form. We could have put ctl as a second function variable, if we had multiple consoles.

Next, the blue text. The “InvokeRequired” property (think of it like a read-only variable) is a nifty shorthand which tells us if the given control was created by the thread we are currently running. As all you GUI programmers know, most actions that update GUI components should be done in the master dispatch loop of the thread that created the GUI. If, for example, this control was created in a random thread which is now finished running, then updating it from another thread will cause all sorts of exceptions at run-time. We could dispatch a new thread for each update operation, but the InvokeRequired property helps us to avoid spawning threads when we don’t need to.

The purple text is run if an Invoke is Required. It calls the Invoke method of ctl, which adds a new event to a thread guaranteed to be valid for modifying ctl. This thread is of type AddToConsoleDel, which we defined earlier as:

delegate void AddToConsoleDel(String msg)

…in other words, as a delegated function with one argument that returns nothing (void). This matches the function signature of our UpdateConsole function. Look back at the purple text, and you’ll see that we call new AddToConsoleDel(UpdateConsole) to  create a new delegate of type AddToConsoleDel that will run the UpdateConsole function when called. The second parameter to Invoke is the argument set that we intend to pass to our AddToConsoleDel delegate. Since UpdateConsole only takes one argument, we only pass in one (msg).

Which leads us to the green text: what happens if we are running in the right thread. If this is the case, we simply append the message string to the Text Box’s existing Text property.

So, the logic of our function is simple. If the thread is capable of modifying the control directly, it just appends the text. Otherwise, it creates a new delegate to the same function, passes along the same method arguments, and asks the given control to Invoke this delegate. That ensures that next time the function is called (by the delegate this time) it has the necessary permission to modify the control. This logic is a bit tricky to catch the first time you see it, but it’s one of the main design patterns of C#. And, it has the benefit of keeping the actual logic of the method in the same place as the invoke logic of the delegate. So we can just call UpdateConsole and not have to care about whether a thread is spun off to do the dirty work.

You can think of a delegate as a pointer to a function, with enhanced type-safety, if that’s any easier for you.
Take a break, grab a coffee, you deserve a break.

Step 3: Passing Messages Along Properly

Our middleware can locate the Eternal Lands server, and it establishes a TCP connection (proven by the fact that it receives two messages in sequence, not just two copies of the same message). We also jumped the gun and added a loopback entry to the client’s list of servers, to allow us to force the client to connect to our program instead of the real server. Now, let’s put that new entry to work, and tie the client and server together so that you can play Eternal Lands with our program as a local proxy.

Open your project in Visual Studio, and right-click on the “Solution” (top-most item) in the “Solution Explorer”. Choose “Add”, then “New Project”, then select “Class Library” and name it Eternal Lands Connector. Press “Ok”. Now, right-click on your main project (“Windows Forms Application 1”, in my case) and choose “Project Dependencies”. In the “Depends on” box, check the “Eternal Lands Connector” project. Click “Ok”.

The purpose of this secondary project is to isolate our code. Once we’ve written the C# necessary to tunnel messages from the client to the server, we’ll then be adding a whole bunch of filtering and display code. We don’t want to accidentally break our message passing code, so we’ll keep in a totally separate project. You could manage this project separately, and manually add the DLL to the main Windows Forms project each time you update it, but Visual Studio’s dependency tracker allows you to do this automatically.

Let’s get started. First, rename Class1.cs to ELMiddleware.cs. We’ll need another file, though (Right-click “Eternal Lands Connector”, choose “Add”, then “New Item…”, then select “Class”) called SleepQueue.cs. This queue manages a list of messages, and stops processing them when the queue is empty. It sets itself to sleep until a new message is received. This signaling approach is far more efficient than constantly checking if the queue has items (spinning). Here’s the code for this class:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Net;
using System.Net.Sockets;
using System.IO;

namespace Eternal_Lands_Connector
{
  public abstract class SleepQueue : IDisposable
  {
    //For threading: allow sleeping when no messages are received, by using an EventWaitHandle
    private Thread t;
    private EventWaitHandle wHandle = new AutoResetEvent(false);
    private Queue<PacketData> waitingMessages = new Queue<PacketData>();

    //Add a packet to the list of waiting messages. 
    // Wake the main thread.
    public virtual void AddPacket(PacketData p)
    {
      lock (waitingMessages)
        waitingMessages.Enqueue(p);
      wHandle.Set();
    }

    //What priority should the message queue thread be? 
    //  Sub-classes can override this to balance performance.
    public virtual ThreadPriority getInitialPriority()
    {
      return ThreadPriority.Normal;
    }

    //Dispose properly
    public void Dispose()
    {
      // Signal the consumer to exit.
      AddPacket(null);
     if (t != null)
        t.Join();
     if (wHandle != null)
        wHandle.Close();
    }

    //Start the queueing process.
    public void startQueueing()
    {
      t = new Thread(new ThreadStart(Handle_Queue));
      t.IsBackground = true;
      t.Priority = getInitialPriority();
      t.Start();
    }

    private void Handle_Queue()
    {
      //Do pre-inits
      Pre_Init_Activity();

      //Listen for queue updates
      PacketData msg = null;
      for (; this.isValid(); )
      {
        //Get the next task
        msg = null;
        lock(waitingMessages)
         {
          if (waitingMessages.Count > 0)
           {
             msg = waitingMessages.Dequeue();
           if (msg == null)
             {
             //Done.
             this.On_Message(msg);
             return;
             }
           }
         }

        //A bit confusing to have two null checks, but it makes sense.
        if (msg == null)
         {
          //No more tasks - wait for a signal
              wHandle.WaitOne();
         } else {
            this.On_Message(msg);
         }
       }

      this.On_Thread_Close();
    }

    //
    //Un-implemented functionality
    //

    //When does this queue exit?
    protected abstract bool isValid();

    //What should we do with each message?
    protected abstract void On_Message(PacketData msg);

    //What should happen when the thread closes? 
    protected abstract void On_Thread_Close();

    //What should happen before the message loop begins? (After the thread is started)
    protected abstract void Pre_Init_Activity();
  }
}

This depends on the PacketData.cs file, which is simply a wrapper on a byte array coupled with an id character:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Eternal_Lands_Connector
{
  public class PacketData
  {
    public PacketData(byte[] data, char from)
    {
      this.data = data;
      this.from = from;
    }

    public byte[] data;
    public char from;
  }
}

The technical details of the Sleep Queue are not important, so long as you realize that:

  1. Every message posted to the queue gets processed.
  2. You can post null to stop processing and end the thread.
  3. The thread takes up no CPU time when there are no messages waiting.
  4. Sub-classes will have to tell the queue when to stop processing (naturally), what to do with messages that are received, and what to do directly before entering and after leaving the message loop.

We now have everything we need to implement the ELMiddleware class, whose purpose is to handle all messages between the client and the server.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Net;
using System.Net.Sockets;
using System.IO;

namespace Eternal_Lands_Connector
{
    //Global
    public delegate void AddPacketDel(PacketData msg);
    public delegate void ShowConnectMsgDel(String msg);
    public delegate void AddToConsoleDel(String msg);
    public delegate void IncrPacketCountDel(int amt);

    public struct ServerInfo
    {
        public String id;
        public String configDir;
        public String address;
        public int port;
        public String description;
        public override String ToString()
        {
            return description;
        }
    }

    public class ELMiddleware : SleepQueue
    {
        //Hooks for controlling the client and server.
        private ELClientHook client;
        private ELServerHook server;

        //Saved
        private ServerInfo currServer;
        private ServerInfo loopback;
        private String elClientDir;
        private ShowConnectMsgDel ShowConnect;
        private AddToConsoleDel UpdateConsole;
        private IncrPacketCountDel IncrPacketCount;
        private int hookThreadID;

        public override void AddPacket(PacketData p)
        {
            //Log
            IncrPacketCount(1);

            //Count
            base.AddPacket(p);
        }

        private bool started = false;
        public bool Started {
            get {
                return started;
            }
        }

        public ELMiddleware(ELClientHook client, ELServerHook server)
        {
            //Save
            this.client = client;
            this.server = server;

            //Hook
            this.client.PacketHook = new AddPacketDel(AddPacket);
            this.server.PacketHook = new AddPacketDel(AddPacket);
        }

        public void startAll(ServerInfo currServer, ServerInfo loopback, String elClientDir, ShowConnectMsgDel ShowConnect, AddToConsoleDel UpdateConsole, IncrPacketCountDel IncrPacketCount, int hookThreadID)
        {
            //Don't start twice
            if (started || client.Started || server.Started)
                return;
            started = true;

            //Save
            this.currServer = currServer;
            this.loopback = loopback;
            this.elClientDir = elClientDir;
            this.ShowConnect = ShowConnect;
            this.UpdateConsole = UpdateConsole;
            this.IncrPacketCount = IncrPacketCount;
            this.hookThreadID = hookThreadID;

            //Start the sleep queues
            base.startQueueing();
        }

        protected override void Pre_Init_Activity()
        {
            //Init Log
            ShowConnect("EL Notifier Init");

            //Start the server
            server.hookServer(currServer, UpdateConsole);

            //Start the client
            client.hookClient(loopback, elClientDir, UpdateConsole, hookThreadID);

            //Spin-wait for both
            while (!server.ReadyForData || !client.ReadyForData)
                Thread.Sleep(5);
            UpdateConsole("\r\nSpin-waiting done");
        }

       protected override void On_Message(PacketData msg)
        {
            //Count
            IncrPacketCount(-1);
            if (msg == null)
                return;

            //Forward this message to the appropriate third party
            if (msg.from == 'C')
            {
                //Send from the client to the server.
                server.sendData(msg.data);
            }
            else if (msg.from == 'S')
            {
                //Send from the server to the client
                client.sendData(msg.data);
            }
            else
                UpdateConsole("\r\nBad message code: " + msg.from);
        }

        protected override bool isValid()
        {
            return client.Started && server.Started;
        }

        protected override void On_Thread_Close()
        {
            //Make sure everything closed nicely
            if (!server.Done)
                server.Done = true;
            if (!client.Done)
                client.Done = true;

            //Done
            started = false;
        }
    }
}

Add two more classes to the project, called ELClientHook.cs and ELServerHook.cs, respectively. Mark each class public, but don’t add any implementation yet.

Let’s review the code for ELMiddleware. At the top, we have some global delegate definitions, presumably used for logging. We also have our ServerInfo struct —if you think this means we’ll be removing a lot of code from Form1.cs then you guessed right. Our ELMiddleware class extends the SleepQueue abstract class. It contains several delegate references, a mysterious hookThreadID, and a reference to our client and server hooks.

Most of the code after this is obvious, and should give you an idea what kind of functionality we’ll need in ELClientHook and ELServerHook. The onMessage() function simply checks the sender of the message, and then ferries the message across to the recipient. We will later add the ability to react to ‘S’ class messages.

Our client and server hooks will need to be able to “take” messages from the middleware, in addition to other miscellaneous features. First, let’s add ELServerHook. Its functionality will be very similar to the previous chapter’s exercises.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.IO;
using System.Net.Sockets;
using System.Threading;

namespace Eternal_Lands_Connector
{
    public class ELServerHook
    {
        private TcpClient tcpclnt;
        private Stream srvStream;
        private ServerInfo srv;

        //Callbacks
        private AddToConsoleDel UpdateConsole;

        private bool started = false;
        public bool Started {
            get {
                return started;
            }
        }

        private bool done;
        public bool Done {
            get {
                return done;
            }
            set {
                done = value;
            }
        }

        private bool readyForData = false;
        public bool ReadyForData {
            get {
                return readyForData;
            }
        }

        private AddPacketDel packetDel;
        public AddPacketDel PacketHook {
            get {
               return this.packetDel;
            }
            set {
                this.packetDel = value;
            }
        }

        public void hookServer(ServerInfo srv, AddToConsoleDel UpdateConsole)
        {
            //Don't start twice
            if (started)
                return;
            started = true;

            //Save
            this.srv = srv;
            this.UpdateConsole = UpdateConsole;

            //Start
            tcpclnt = new TcpClient();
            UpdateConsole("Connecting.....");

            Thread t = new Thread(new ThreadStart(Check_Server));
            t.IsBackground = true;
            t.Start();
        }

        public void sendData(byte[] data)
        {
            if (!readyForData)
                throw new Exception("Server is not ready for messsages");

            //Send the dat down this stream. Technically, this occurs in the middleware's thread,
            //   so it won't conflict with any data we're reading at the same time.
            try {
                srvStream.Write(data, 0, data.Length);
            } catch (Exception ex) {
                UpdateConsole("\r\nException writing to server: \r\n   " + ex.ToString());
            }
        }

        private void Check_Server()
        {
            //Connect to the server's TCP port.
            tcpclnt.Connect(srv.address, srv.port);
            readyForData = true;
            UpdateConsole("Connected");

            //Try reading a bit
            srvStream = tcpclnt.GetStream();
            byte[] msg = new byte[1024];
            done = false;
            for (; !done; )
            {
                int amtRead;
                try {
                    amtRead = srvStream.Read(msg, 0, msg.Length);
                } catch (IOException) {
                    UpdateConsole("\r\nIOException; closing Server");
                    amtRead = 0;
                }

                if (amtRead != 0)
                {
                    //Make a new packet, add it to the middleware's queue
                    byte[] pk = new byte[amtRead];
                    Array.Copy(msg, pk, amtRead);
                    PacketHook(new PacketData(pk, 'S'));
                }
                else
                {
                    UpdateConsole("\r\nRead zero; closing Server");
                    done = true;
                }
            }

           //Must close BOTH
            srvStream.Close();
            tcpclnt.Close();

            //Ready to go again?
            UpdateConsole("\r\nServer Stopped");
            Console.WriteLine("Server hook stopped");
            started = false;
        }
    }
}

The implementation is almost exactly what we had before. The call to srvStream.Read() is blocking, by the way —no Sleep Queue is necessary.

The ELClientHook implementation is much more elusive:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.IO;
using System.Net.Sockets;
using System.Text;
using System.Text.RegularExpressions;
using System.Diagnostics;
using System.Threading;
using System.Windows.Forms;
using System.Runtime.InteropServices;

namespace Eternal_Lands_Connector
{
    //Just a helper class; no threads.
    public class ELClientHook
    {
        //Const
        public const string SERVER_ADDR = "127.0.0.1";
        public const int SERVER_PORT = 5421;

        private ServerInfo loopback;
        private String elClientDir;
        private TcpListener listener;
        private Stream clientStr;

        private Form mainForm;
        private int hookThreadID;
        private bool elDone = false;

        private bool started = false;
        public bool Started {
            get {
                return started;
            }
        }

        private bool done;
        public bool Done {
            get {
                return done;
            }
            set {
                done = value;
            }
        }

        private bool readyForData = false;
        public bool ReadyForData {
            get {
                return readyForData;
            }
        }

        private AddPacketDel packetDel;
        public AddPacketDel PacketHook {
            get {
                return this.packetDel;
            }
            set {
                this.packetDel = value;
            }
        }

        //Callbacks
        private AddToConsoleDel UpdateConsole;

        public void hookClient(ServerInfo loopback, String elClientDir, AddToConsoleDel UpdateConsole, int hookThreadID)
        {
            //Don't start twice
            if (started)
                return;
            started = true;

            //Save
            this.loopback = loopback;
            this.UpdateConsole = UpdateConsole;
            this.elClientDir = elClientDir;
            this.hookThreadID = hookThreadID;

            //Start background client
            Thread t = new Thread(new ThreadStart(Start_Client));
            t.IsBackground = true;
             t.Start();
        }

        public void sendData(byte[] data)
        {
            if (!readyForData)
                throw new Exception("Server is not ready for messsages");

            //Send the dat down this stream. Technically, this occurs in the middleware's thread,
            //   so it won't conflict with any data we're reading at the time.
            try {
                clientStr.Write(data, 0, data.Length);

                //Logging code - uncomment for development
                /*StringBuilder sb = new StringBuilder();
                sb.Append("\r\nClient Write: [");
                foreach (byte b in data)
                    sb.Append(Convert.ToString(b, 16).PadLeft(2, '0') + ":");
                sb.Append("]");
               UpdateConsole(sb.ToString());*/
            } catch (Exception ex) {
                UpdateConsole("\r\nException writing to client: \r\n   " + ex.ToString());
            }
        }

        private void EL_Stopped(Object sender, EventArgs e)
        {
            elDone = true;
        }

        //Get our handle (no idea why this requires an invoke... maybe it creates it?)
        private delegate IntPtr GetFormHandleDel(Form f);
        private IntPtr GetFormHandle(Form f)
        {
            if (f.InvokeRequired)
                return (IntPtr)f.Invoke(new GetFormHandleDel(GetFormHandle), f);
            else
                return f.Handle;
        }

        private bool findMainForm(int[] allowedProcessIDs, String matcherRegex, int hookThreadID)
        {
            return true;
        }

        private void EL_Window_Activate()
        {
            Console.WriteLine("EL Client Window Activated");
        }

        private void EL_Window_Deactivate()
        {
            Console.WriteLine("EL Client Window Deactivated");
        }

        private void Start_Client()
        {
            //Start the socket
            listener = new TcpListener(IPAddress.Parse(SERVER_ADDR), SERVER_PORT);
            listener.Start();
           UpdateConsole("\r\nLoopback server running at: " + listener.LocalEndpoint);

            //Start the EL client program with the server specified as "loopback"
            UpdateConsole("\r\nStarting EL Client...");
            ProcessStartInfo ps = new ProcessStartInfo();
            ps.FileName = elClientDir + "\\el.exe";
            ps.Arguments = loopback.id;
            ps.WorkingDirectory = elClientDir;
            ps.ErrorDialog = true;
            Process p = Process.Start(ps);
            elDone = false;
            p.Exited += new EventHandler(EL_Stopped);

            //Get the full-screen window for this application
            mainForm = null;
            while (mainForm == null)
            {
                int[] validIDs = new int[p.Threads.Count];
                {
                    int id = 0;
                    foreach (ProcessThread t in p.Threads)
                        validIDs[id++] = t.Id;
                }

                if (findMainForm(validIDs, "Eternal Lands", hookThreadID))
                    break;
                Thread.Sleep(5);
            }
            UpdateConsole("\r\nClient spin-waiting done");

            //Wait for the client to try to connect to us
            TcpClient socket = listener.AcceptTcpClient();
            clientStr = socket.GetStream();
            readyForData = true;
            UpdateConsole("\r\nEL Client tried to connect to us: " + socket.Client.RemoteEndPoint);

            //Continually read from the client. This is the main client update thread.
            byte[] msg = new byte[1024];
            done = false;
            for (; !done; )
            {
                //Is the client still running?
                if (elDone)
                {
                    UpdateConsole("\r\nGame closed; closing Client");
                    done = true;
                }
                else
                {
                    int amtRead;
                    try
                    {
                       amtRead = clientStr.Read(msg, 0, msg.Length);
                    }
                    catch (IOException)
                    {
                        UpdateConsole("\r\nIOException; closing Client");
                        amtRead = 0;
                    }
                    if (amtRead != 0)
                    {
                       //Make a new packet, add it to the middleware's queue
                        byte[] pk = new byte[amtRead];
                        Array.Copy(msg, pk, amtRead);
                        PacketHook(new PacketData(pk, 'C'));
                    }
                    else
                    {
                        //We're done
                        UpdateConsole("\r\nRead zero; closing Client");
                        done = true;
                    }
                }
            }

            //Have to close all three
            clientStr.Close();
            socket.Close();
            listener.Stop();

            //Ready to go again?
            UpdateConsole("\r\nClient Stopped");
            Console.WriteLine("Client hook stopped");
            started = false;
        }
    }
}

You might get an error with the System.Windows.Forms using directive; simply right-click on the “References” section of your Connector’s Project file and choose “Add Reference”. Then, in the “.NET” tab, browse down to “System.Windows.Forms” and click “Ok”. By default, a library project does not have access to the Forms dll, for reasons that remain a mystery to me.

A quick browse through this code reveals that our fake middleware server will be situated at 127.0.0.1, on port 5421. The sendData() function is similar to the server’s copy; however, I’ve included some logging code for your convenience. If you uncomment this, it will print a properly-formatted hexadecimal representation of each message sent from the server to the client. This will allow you to manually debug unknown or tricky messages, and to determine how you should react to them.

The findMainForm() function is currently empty; it will later be used to determine which window is actually hosting the EL client. The only other function of interest is the StartClient() function. Note that we use a ProcessStartInfo object to handle starting a process:

1.We have to set the working directory to the Eternal Lands folder, otherwise the process won’t be able to find its resources (images, sounds, etc.).
2.We start the process el.exe. This, of course, is Windows-specific.
3.We start this process with the argument loopback. In other words, we call “el.exe loopback” from the command line, allowing us to choose which server to connect to.

The Thread-finding code which follows this is messy; just ignore it. Basically, our middleware program will have multiple running threads, and we need to figure out which one is hosting the EL Client’s window. That way, we can detect when it has focus, and avoid showing messages in that case.

Note that, like the server, the client continually reads from the middleware until no more data is sent. You’ll have to change Form1’s code to actually use this new middleware, of course.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using Eternal_Lands_Connector;
using System.Text.RegularExpressions;
using System.IO;
using System.Threading;
using System.Net.Sockets;

namespace WindowsFormsApplication1
{
    public partial class Form1 : Form
    {
        //Count leftover packets. A good first-warning if something goes wrong.
        private int packetCount;

        //Our hooks
        private ELServerHook serverHook;
        private ELClientHook clientHook;
        private ELMiddleware middleWare;

        //Also...
        private ServerInfo loopbackInfo;
        private string srvLine = "loopback        main            " + ELClientHook.SERVER_ADDR + "               " + ELClientHook.SERVER_PORT + "    Loopback adapter for use with the EL Notifier";

        public Form1()
        {
            InitializeComponent();

            //Initialize connectors.
            serverHook = new ELServerHook();
            clientHook = new ELClientHook();
            middleWare = new ELMiddleware(clientHook, serverHook);
        }

        private void ReadELFiles(object sender, EventArgs e)
        {
            //Reload an entirely new list of servers.
            cmbServers.Items.Clear();

            //Don’t load anything if we specified an invalid directory.
            if (!File.Exists(txtPathToELFolder.Text + "\\" + "el.exe"))
            {
                txtConsole.Text = "Error: el.exe doesn't exist in EL folder";
                return;
            }
            else if (!File.Exists(txtPathToELFolder.Text + "\\" + "servers.lst"))
            {
                txtConsole.Text = "Error: servers.lst doesn't exist in EL folder";
                return;
            }

            //Open our servers file and read it.
            bool foundLoopback = false;
            StreamReader srvFile = new StreamReader(txtPathToELFolder.Text + "\\" + "servers.lst");
            while (!srvFile.EndOfStream)
            {
                String line = srvFile.ReadLine();
                if (line == null)
                    break;

                //Coment? Empty?
                line = line.Trim();
                if (line.StartsWith("#") || line.Length == 0)
                    continue;

                //Read:
                //ID, Config_Dir, Address, Port, Description
                String regex = "([^ \t]+)[ \t]+([^ \t]*)[ \t]+([^ \t]*)[ \t]+([^ \t]*)[ \t]+(.*)";
                Match m = Regex.Match(line, regex);
                ServerInfo sInf = new ServerInfo();
                if (m.Success && m.Groups.Count == 6)
                {
                    sInf.id = m.Groups[1].Value;
                    if (sInf.id.Equals("loopback"))
                        foundLoopback = true;
                    sInf.configDir = m.Groups[2].Value;
                    sInf.address = m.Groups[3].Value;
                    try
                    {
                        sInf.port = Int32.Parse(m.Groups[4].Value);
                    }
                    catch (FormatException)
                    {
                        //Don't add
                        continue;
                    }
                    sInf.description = m.Groups[5].Value;
                    cmbServers.Items.Add(sInf);

                    //Loopback?
                    if (sInf.id.Equals("loopback"))
                        loopbackInfo = sInf;
                }
            }
            srvFile.Close();

            //Do we need to add our own local server?
            if (!foundLoopback)
            {
                //Copy existing server file
                StreamReader inFile = new StreamReader(txtPathToELFolder.Text + "\\" + "servers.lst");
                StreamWriter outFile = new StreamWriter(txtPathToELFolder.Text + "\\" + "servers.lst.new");
                for (String line = inFile.ReadLine(); line != null; line = inFile.ReadLine())
                    outFile.WriteLine(line);
                inFile.Close();

                //Add our own category
                outFile.WriteLine(srvLine);

                //Close, swap, notify.
                outFile.Close();
                File.Delete(txtPathToELFolder.Text + "\\" + "servers.lst");
                File.Move(txtPathToELFolder.Text + "\\" + "servers.lst.new", txtPathToELFolder.Text + "\\" + "servers.lst");
                ShowConnectMsg("Created loopback entry.");

                //Add a new item to the list
                loopbackInfo = new ServerInfo();
                loopbackInfo.id = "loopback";
                loopbackInfo.configDir = "main";
                loopbackInfo.address = ELClientHook.SERVER_ADDR;
                loopbackInfo.port = ELClientHook.SERVER_PORT;
                cmbServers.Items.Add(loopbackInfo);
            }

            if (cmbServers.Items.Count > 0)
                cmbServers.SelectedIndex = 0;

            //Enable the remainder of our form
            cmbServers.Enabled = true;
            btnConnect.Enabled = true;
        }

        private void ConnectToServer(object sender, EventArgs e)
        {
            if (middleWare.Started || clientHook.Started || serverHook.Started)
                return;

            //Count
            packetCount = 0;

            //Get our currently-selected server and start it.
            ServerInfo srv = (ServerInfo)cmbServers.SelectedItem;
            middleWare.startAll(srv, loopbackInfo, txtPathToELFolder.Text, new ShowConnectMsgDel(ShowConnectMsg), new AddToConsoleDel(UpdateConsole), new IncrPacketCountDel(IncrPacketCount), Thread.CurrentThread.ManagedThreadId);
        }

        //Delegates and implementation details
        private void ShowConnectMsg(String msg)
        {
            Control ctl = txtConsole;
            if (ctl.InvokeRequired)
                ctl.Invoke(new ShowConnectMsgDel(ShowConnectMsg), msg);
            else
                ctl.Text = msg;
        }

        private void UpdateConsole(String msg)
        {
            Control ctl = txtConsole;
            if (ctl.InvokeRequired)
                ctl.Invoke(new AddToConsoleDel(UpdateConsole), msg);
            else
                ctl.Text += msg;
        }

        private void IncrPacketCount(int amt)
        {
            packetCount += amt;
            SetText(lblZero, packetCount + "");
        }

        private delegate void SetTextDel(Control ctl, String msg);
        private void SetText(Control ctl, String msg)
        {
            if (ctl.InvokeRequired)
                ctl.Invoke(new SetTextDel(SetText), ctl, msg);
            else
                ctl.Text = msg;
        }
    }
}

Right-click on the References section in your main Forms project and choose “Add Reference”. Then, in the “Projects” tab, choose “Eternal Lands Connector” and hit “Ok”. This will allow you to use our class library.

The source code is not too complex, and the effect is easy enough to describe: we simply intercept and then deliver all messages from the client to the server, and from the server to the client. At the moment, this is essentially a whole lot of work for nothing. However, now that all messages are passing through our program (our source code, you might say), you can easily  tag messages of a specific type, and prompt the user after further processing them. So, this was all “legwork”.

Our middleware in action, connecting to the PK server.

Our middleware in action, connecting to the PK server.

Congratulations, we have now bamboozled both end points. In the next section, we’ll consider how to filter messages without requiring a re-compile of the Connector each time we change our minds about which messages to filter. We’ll also get you started processing Eternal Lands’s images to brighten up our notifier.

Step 4: Filtering Messages (Outside the Core)

At this point, you should have a pretty good idea how to intercept messages of a certain type. Before you dash too far into it, though, may I remind you that one of the reasons we used C# in the first place was to avoid bugs that corrupted our communication layer. We should add the following to the communication core:

  1. Allow our main Forms class to “register” any number of listeners with the core.
  2. Before sending a message (to the client), check if it matches any of the listeners’ patterns. If it does, send a copy of the message to the matched listener.
  3. Ensure that the core runs in a higher-priority thread than the listeners’ implementations. According to the rules of Windows thread priorities, this ensures that the listeners cannot starve the core for cycles.

This way, no amount of chicanery by the listeners can cause the core to malfunction. (And remember, malfunctions are BAD, since we’re not the ones paying for the EL servers.)

Open up ELMiddleware.cs, and add the following member variable to the class:

//An additional matcher

private MessageMatcher msgMatch;

We’ll define MessageMatcher later; for now, scroll down to the definition of startAll, and change it to read:

public void startAll(ServerInfo currServer, MessageMatcher msgMatch, ServerInfo loopback, String elClientDir, ShowConnectMsgDel ShowConnect, AddToConsoleDel UpdateConsole, IncrPacketCountDel IncrPacketCount, int hookThreadID)

Next, somewhere in the body of startAll, save the MessageMatcher that we passed in to the function:

this.msgMatch = msgMatch;

Make sure you start it too; after “base.startQueueing()” add:

msgMatch.startQueueing();

And finally, scroll down until you find the line:

client.sendData(msg.data);

…and add the following underneath it:

//Send the data to an additional thread for logging, parsing, etc.

msgMatch.AddPacket(msg);

That code should speak for itself; our MessageMatcher class will centralize message processing, and the AddPacket method is called for each server message. We do not filter client messages, because it seems that these originate mostly from the user’s actions (which are always known by the user –and must be confirmed by the server, regardless.)

Here, then, is the definition of the MessageMatcher class. Add it to your project (to the Eternal_Lands_Connector project, that is).

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.Threading;

using System.Net;

using System.Net.Sockets;

using System.IO;

namespace Eternal_Lands_Connector

{

public class MessageMatcher : SleepQueue

{

/// <summary>

/// The core of our message matcher

/// </summary>

private Dictionary<Int32, List<MessageHandler>> handlers = new Dictionary<Int32, List<MessageHandler>>();

public void addHandler(MessageHandler h)

{

if (!handlers.ContainsKey(h.getProtocol()))

handlers[h.getProtocol()] = new List<MessageHandler>();

handlers[h.getProtocol()].Add(h);

Console.WriteLine(“Dictionary size: “ + handlers.Count + ”   entries[" + h.getProtocol() + "] –> “ + handlers[h.getProtocol()].Count);

}

public AddToConsoleDel UpdateConsole;

protected override void On_Message(PacketData msg)

{

if (msg == null)

return;

//Now, segment into multiple messages

for (int currID = 0; currID < msg.data.Length; )

{

int protocol = msg.data[currID++];

int size = msg.data[currID++] | (msg.data[currID++] << 8);

//Make a new data array

int count = size – 1;

byte[] b = new byte[count];

Array.Copy(msg.data, currID, b, 0, count);

currID += count;

//Match the first relevant handler

if (handlers.ContainsKey(protocol))

{

foreach (MessageHandler h in handlers[protocol])

{

if (h.performMatch(protocol, b, new Dictionary<string, object>()))

{

break;

}

}

}

}

}

/// <summary>

/// Somwhat less-interesting features

/// </summary>

private bool valid = true;

public bool IsValid

{

get

{

return valid;

}

set

{

valid = value;

}

}

//We make this its own class so we can change its priority.

public override System.Threading.ThreadPriority getInitialPriority()

{

return ThreadPriority.BelowNormal;

}

protected override bool isValid()

{

return valid;

}

protected override void On_Thread_Close()

{

//Nothing to do here

}

protected override void Pre_Init_Activity()

{

//Nothing to do here

}

}

}

The MessageHandler class hasn’t been created yet, but let’s just say that it matches all messages first on a “protocol” and then on a more complex “performMatch” query. All of these MessageHandlers are stored indexed by protocol in a hash table. This ensures that we can add lots of processing handlers without slowing down the pattern matching.

Finally, note that MessageMatcher is a SleepQueue, which means that even if we do add far too many handlers of the same protocol (say, one for each buddy in our buddy list?), only our matching will be delayed –the core won’t be.

You might be wondering about the structure of Eternal Lands’s messages. From the processing loop in MessageMatcher, you can tell that every message contains a protocol byte as the first item in the message. The next two bytes describe the size of the remaining data. This is more important than you might think: Eternal Lands often sends multiple messages bunched together, and you can only tell where one ends and the next begins by processing the size bytes. Anything after the first three bytes is dependant on the protocol of the message. Also, for some unknowable reason, the actual size of the remaining data is size-1.

Now, make a class called MessageHandler, in the Eternal_Lands_Connector project:

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

namespace Eternal_Lands_Connector

{

public abstract class MessageHandler

{

//All message handlers match on a protocol

public abstract int getProtocol();

//Within a given protocol, does this handler match?

//  This should also perform an action

//  Decorations used to avoid multiple calculations

public abstract bool performMatch(int protocol, byte[] data, Dictionary<String, Object> decorations);

}

}

Were you to compile your project, you would notice that Form1’s call of middleWare.startAll() is missing an argument: MessageMatcher. Before we go passing in an empty class, let’s try thinking about the types of MessageHandlers we might need. Well, the most basic type will simply match based on the protocol, and defer further processing till later. Then, we might want to match text messages sent from the server, using such things as the color of the text, the channel it was sent to, and a regular expression of what the text itself must contain.

First, make a folder (in the main Forms project) called matching. We will put three classes in there: Constants.cs, BasicHandler.cs, and TextHandler.cs. First, let’s cover Constants.cs:

public class Constants

{

public enum Protocols

{

RAW_TEXT = 0,

SEND_PARTIAL_STAT = 49,

GET_NEW_INVENTORY_ITEM = 21,

}

public enum Colors

{

c_green1 = 3,

c_red1 = 0,

}

public enum Channels

{

CHAT_SERVER = 3,

}

}

This class simply contains some useful constants used to identify channels, colors, and protocols. We’ve included only the bare minimum; please add any constants as your message matching needs expand.

Now, BasicHanlder.cs:

class BasicHandler : Eternal_Lands_Connector.MessageHandler

{

public delegate void OnMatchDel(byte[] data);

private int protocol;

private OnMatchDel OnMatch;

public BasicHandler(int protocolMatch, OnMatchDel OnMatch)

{

this.protocol = protocolMatch;

this.OnMatch = OnMatch;

}

public override int getProtocol()

{

return protocol;

}

public override bool performMatch(int protocol, byte[] data, Dictionary<string, object> decorations)

{

//Quick initial match

if (protocol != getProtocol())

return false;

//Always match

OnMatch(data);

return true;

}

}

The unfortunate reality is that this class is so simple that it makes understanding it tricky. It’s very simple:

  1. The constructor takes a protocol value to match, and an OnMatch delegate to perform when that protocol matches. It saves these.
  2. The getProtocol() method exists so that our MessageMatcher can store this MessageHandler in the correct hash table entry.
  3. The performMatch() function always returns true, after performing the OnMatch delegate, assuming the protocol byte matches.

Without thinking too much about it, let’s add the code for TextHandler.cs and compare the two:

public class TextMessage

{

public int channel;

public int color;

public String text;

}

public class TextHandler : Eternal_Lands_Connector.MessageHandler

{

private const String KEY_DECORATIONS = “TextHandler_Decorations”;

public delegate void OnMatchDel(TextMessage m);

private Regex msgPattern;

private int msgColor;

private int msgChannel;

private OnMatchDel OnMatch;

//-1 for “all” on color or channel

public TextHandler(String msgPattern, int color, int channel, OnMatchDel OnMatch)

{

this.msgPattern = new Regex(msgPattern);

this.msgColor = color;

this.msgChannel = channel;

this.OnMatch = OnMatch;

}

public override int getProtocol()

{

return (int)Protocols.RAW_TEXT;

}

public override bool performMatch(int protocol, byte[] data, Dictionary<String, Object> decorations)

{

//Quick check; protocol

if (protocol != getProtocol())

return false;

//Get

if (!decorations.ContainsKey(KEY_DECORATIONS))

{

//Construct

TextMessage t = new TextMessage();

t.channel = data[0];

t.color = data[1] – 127;

//Parse Text

int count = data.Length – 2;

t.text = System.Text.ASCIIEncoding.ASCII.GetString(data, 2, count);

//Later, of course, we’ll have to deal with things like embedded color tags, etc.

//Save

decorations[KEY_DECORATIONS] = t;

}

TextMessage message = (TextMessage)decorations[KEY_DECORATIONS];

//Match -Stage 1

if ((msgChannel != -1 && message.channel != msgChannel) || (msgColor != -1 && message.color != msgColor))

return false;

//Does the text match?

if (!msgPattern.IsMatch(message.text))

return false;

//It matches!

OnMatch(message);

return true;

}

}

You’ll notice that I didn’t include the using directives or the enclosing namespace. Whatever Visual Studio provides as defaults will work in this case. You’ll have to add two additional using directives, however. Note the second one; it assigns an alias to an enum within our Constants class. This allows us to simply reference Protocols.RAW_TEXT, instead of the more cumbersome Constants.Protocols.RAW_TEXT.

using System.Text.RegularExpressions;

using Protocols = WindowsFormsApplication1.matching.Constants.Protocols;

We’re now ready to create some message handlers and log their output. Go to your main form’s ConnectToServer method, and add the following lines after your various checks for “Started”

//Make a Message Matcher to handle all our special cases

MessageMatcher match = new MessageMatcher();

match.UpdateConsole = UpdateConsole;

//Add some filters

TextHandler h1 = new TextHandler(“You stopped harvesting.”, (int)Colors.c_red1, (int)Channels.CHAT_SERVER, new TextHandler.OnMatchDel(On_Harvest_Stop));

match.addHandler(h1);

BasicHandler h2 = new BasicHandler((int)Protocols.GET_NEW_INVENTORY_ITEM, new BasicHandler.OnMatchDel(Got_Item));

match.addHandler(h2);

You will also need some using directives:

using WindowsFormsApplication1.matching;

using Protocols = WindowsFormsApplication1.matching.Constants.Protocols;

using Colors = WindowsFormsApplication1.matching.Constants.Colors;

using Channels = WindowsFormsApplication1.matching.Constants.Channels;

…the “WindowsFormsApplication1” should be the name of your project; you should be able to resolve naming conflicts like this on your own. Finally, add your MessageMatcher into the call to startAll:

middleWare.startAll(srv, match, loopbackInfo, txtPathToELFolder.Text, new ShowConnectMsgDel(ShowConnectMsg), new AddToConsoleDel(UpdateConsole), new IncrPacketCountDel(IncrPacketCount), Thread.CurrentThread.ManagedThreadId);

The only thing left to do is to implement On_Harvest_Stop and Got_Item. Here are the methods:

private void On_Harvest_Stop(TextMessage tm)

{

//Show message:

UpdateConsole(“\r\nYou have stopped harvesting.”);

}

private void Got_Item(byte[] data)

{

//Decode our message

int image_id = data[0] | (data[1]<<8);

int quantity = data[2] | (data[3]<<8) | (data[4]<<16) | (data[5]<<32);

int pos = data[6];

uint flags = data[7];

//Show message:

UpdateConsole(“\r\nYou got “ + quantity + ” item(s): “ + image_id);

The On_Harvest_Stop method simply informs the player that he or she has stopped harvesting. At this point, all we have to do is show a window and we’d be at the same point we were with the previous approach. (We’ll spice that up in the next tutorial). The Got_Item method gets the image_id, quantity, position, and flags for the item the user picked up (through harvesting, from a drop bag, whatever) and informs the user of these values. We’ll expand on this in the next section. How do we get this data? Read the Eternal Lands source code –it’s pretty easy to find what you want. For example, a little digging reveals the purpose of the flags byte:

is_resource        = binary(0010)  //Can be used to manufacture

is_reagent         = binary(0001)  //Can be used in magic

use_with_inventory = binary(1000)  //Item can be used with inventory

is_stackable       = binary(0100)  //The item is stackable

Moving on, compile and run your code, log in to the main server, and harvest something. Alt+Tab to our C# window and wait for the “You have stopped harvesting” message. Useful, eh? You’ll also see some “Got Item” messages.

post_9_pt1_both_filters_work

Several Item and Harvest Messages

You should take some time to study these messages. First of all, you’ll notice that the quantity of a “got” item is the total number you now have, not the number you just picked up. Second, you’ll see that one of the items we were harvesting (id 35) was harvested at 2x the normal rate. This is because we were at one of the 2x spots for that item.

Step 5: Bonus! Fun With Images

At this point, we’re basically done with phase one of our project. We can hook the server and filter messages by a variety of criteria (without recompiling the connection code). But before we call it quits, there’s one thing that’s bothering me: what exactly is item 35? Unfortunately, there’s no easy way to tell, as the Eternal Lands client simply deals with images and identifiers. But maybe we can at least show a picture of the item we just picked up?

If you dig a bit in the source code, you’ll find that the following code is used for drawing items:

cur_item_img=item_list[pos].image_id%25;

u_start=0.2f*(cur_item_img%5);

u_end=u_start+(float)50/256;

v_start=(1.0f+((float)50/256)/256.0f)-((float)50/256*(cur_item_img/5));

v_end=v_start-(float)50/256;

//get the texture this item belongs to

this_texture=get_items_texture(item_list[pos].image_id/25);

get_and_set_texture_id(this_texture);

glBegin(GL_QUADS);

if(mini)

draw_2d_thing(u_start,v_start,u_end,v_end,mouse_x-16,mouse_y-16,mouse_x+16,mouse_y+16);

else

draw_2d_thing(u_start,v_start,u_end,v_end,mouse_x-25,mouse_y-25,mouse_x+25,mouse_y+25);

glEnd();

Converting this to pseudo-code, we get:

image_id = curr_item_img + (25*item_texture_num)

curr_item_img = u_comp + (5*v_comp)

u_start = 0.2*u_comp

u_end = u_start + 0.195  //Width is roughly in .2 increments

v_start = 1.00076 – 0.195 * v_comp

v_end = v_start – 0.195

A shrewd observer (or one accustomed to OpenGL) will notice that this code assumes Cartesian coordinates, so we’d have to flip the v_* variables.

Before we convert this to C#, browse to the Eternal Lands directory, and look in the “textures” sub-folder. Then, look at any of the “items*.bmp” files. Hmm… these images look mighty familiar…

post_9_pt1_items3

A 5x5 Grid of Eternal Lands Items

Here’s the code to open one of these 5×5 items files, and read each separate image into a Bitmap object. Add this to your main Form’s ReadELFiles method, right before setting cmbServers.Enabled to true.

//Now load our bitmaps from the textures folder

for (int img_texture_id = 0; true; img_texture_id++)

{

//Load this image

String path = txtPathToELFolder.Text + “\\” + “textures” + “\\” + “items” + (img_texture_id + 1) + “.bmp”;

if (!File.Exists(path))

break;

Bitmap patch = new Bitmap(path);

//Loop through all possible images

for (int curr_item_img = 0; curr_item_img < 25; curr_item_img++)

{

//Construct a composite id

int composite_id = curr_item_img + 25 * img_texture_id;

//Get the sub-images’ co-ordinates

float u_start = 0.2F * (curr_item_img % 5);

float u_end = u_start + (float)50 / 256;

float v_start = (1.0f + ((float)50 / 256) / 256.0f) – ((float)50 / 256 * (curr_item_img / 5));

float v_end = v_start – (float)50 / 256;

//Convert to pixels, non-Cartesian

int xStart = (int)(u_start * 256);

int xEnd = (int)(u_end * 256);

int yStart = 256 – (int)(v_start * 256);

int yEnd = 256 – (int)(v_end * 256);

//Draw this into its own bitmap

Bitmap pic = new Bitmap(xEnd – xStart, yEnd – yStart);

Graphics g = Graphics.FromImage(pic);

g.DrawImage(patch, new Rectangle(0, 0, xEnd – xStart, yEnd – yStart), new Rectangle(xStart, yStart, xEnd – xStart, yEnd – yStart), GraphicsUnit.Pixel);

itemPics[composite_id] = pic;

}

}

The code is pretty self-explanatory; it simply reads all items*.bmp files until one doesn’t exist. For each of the 25 items, it applies the segmentation algorithm. It does require one more variable, added at the top of the class:

private Dictionary<Int32, Bitmap> itemPics = new Dictionary<int, Bitmap>();

Now, let’s use this! Add the following code to the end of Got_Item, replacing the existing call to UpdateConsole:

UpdateGotImage(image_id);

Now, implement our delegate and method call, as usual:

private delegate void UpdateGotImageDel(int image_id);

private void UpdateGotImage(int image_id)

{

if (justGotItem.InvokeRequired)

justGotItem.Invoke(new UpdateGotImageDel(UpdateGotImage), image_id);

else

{

justGotItem.Image = itemPics[image_id];

}

}

This references a GUI component that doesn’t exist yet. Add a PictureBox to your Form (anywhere it fits) and set its name to justGotItem and its size to 50,50. Sorry for approaching this in a rather roundabout fashion, but I’m sure you were able to keep up. Compile and run your code. Now, pay attention to your C# window every time you get a new item.

post_9_pt1_catching_roses

Harvesting Roses; Game Proves It's Working

Proof that this actually works is somehow so much more satisfying once images are involved. I’m at a loss to explain it.

The possibilities from here on in are really endless. What about catching Private Messages from players, and funneling them into GTalk through their API? How about updating your Eternal Lands blog with the item you are harvesting or making, in real-time? What about saving the value returned each time you use an Astrology Indicator, and then charting the results to see how your luck fares over time? All of these things are possible, and none (as near as I can tell) violate the terms of use of Eternal Lands. But make sure you post on the EL forum, if you’re not sure. It’s better to get permission rather than get banned.

I regret that I’ve only posted half the code I’ve written at this point. It just takes so long to re-write and re-test everything. I’m a bit tired of all this right now; next time I feel like writing 50 pages of text, though, I’ll finally show you how to soup up your message interface. Consider the following screenshots as teasers. Note the cool “dissolve in” effect, and the “maximize” and “close” buttons (I edited the image; normally they only highlight when you mouse over them.) All of this code works; I just need to document it, and hook up the notifications to real messages (no, I did not get three enrichment stones in a row. :P )

post_9_pt1_system_tray

Nice System Tray Icon

post_9_pt1_message_stack

Version 2.0 Has a Fancy, Animating Notification Stack

Final Discussion

Eternal Lands is one of my favorite online games, and yet like all cross-platform games, it doesn’t feel tightly integrated into Windows. One benefit of them opening the source to developers is that we can now hack around and improve the code as we see fit, distributing our changes to benefit everyone. In this article, I showed you how to enhance the notifications provided by Eternal Lands, to make harvesting of high-level items less tedious. By using a high-level language (C#) with low-level hooks into Win32 networking, we achieve both power and speed, with the potential to add nice animations and Windows-specific polish as well. Best of luck with your own hacks!

License – Preamble

The current code cannot be used to abuse the terms of use of Eternal Lands, and I’ve chosen a carefully-worded license to prevent further modifications of this code from being used for abuse. The Eternal Lands developers trusted us with their code, and I intend to honor that trust. Please do the same. The following license applies to source code, compiled binaries, and this article, describing how to make the program from scratch. All future source and binary releases (as well as updates to this article) will remain under this license.

License

Copyright (c) 2009 Seth N. Hetu

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The Software may not be modified to abuse the Terms and Conditions of the Eternal Lands game, as expressed in its license (http://www.eternal-lands.com/page/license.txt).

The Software may not be used to inject events to, nor remove messages from, the regular stream of packets sent between the Eternal Lands client and the Eternal Lands server.

All messages sent from the Eternal Lands client to the Eternal Lands server and vice-versa must arrive at their intended destinations without delay and without reordering or modifying them.

The Thread Priority of the event handler used by the Software may not be modified; moreover, any threads created that interact with the Software in any way MUST have a lower Thread Priority than that of the event handler.

If, at any time, an Eternal Lands moderator (as listed at: http://www.eternal-lands.com/page/developers.php?#mods) or Eternal Lands creator (http://www.eternal-lands.com/page/developers.php?#creators) requests that the Software or any specific derivative work be removed, for any reason, this demand must be met, and all source code and compiled code for the Software must be removed from public display. (Aside: We would ask moderators to be reasonable when demanding that the Software be removed.) The Software and its sources may be restored to public display only with the permission of an Eteranl Lands moderator. In the case of a dispute between moderators and creators, the creators are given preference.

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

June 24, 2009

[LZ-3] Intermediate Hyrule Magic

Blog Info: Staffman of the week goes to either Ubuntu or Fedora —whichever OS decided to corrupt the first block of this file, forcing me to re-type the entire thing.

Back-story: Sometimes there’s no quaint little project (like LZ-2) that gets you involved in the software; you just have to force yourself to accomplish stock goals. That’s what we’ll be doing in today’s lesson: solving a series of moderately difficult, realistically useful tasks related to Zelda hacking.

Goal: Walk through some typical usage scenarios with Hyrule Magic.

Don’t forget your hash:
1a74468291b02729329dd1357afb45af *Zelda3.smc

Table Of Contents

1. How Do I Edit “Link’s Graphics” In Hyrule Magic?

2. How Do I Save An Image in 16-Color Format?

3. How Do I Edit Other Graphics? How Do I Use Zcompress and YY-CHR?

4. How Do I Edit Overworld Tiles? What’s This 16×16 Tile Nonsense?

5. Lunch~

6. How Do I Change Flute Locations?

7. How Do I Edit The Game’s 3-D Graphics? How Complex Can These Graphics Be?

8. How Do I Change The Music For An Area?

9. How Do I Make An Enemy Drop a Key? How Do I Choose Which Enemy Drops the Key?

10. How Do I Add a Nifty Logo To The Title Screen?

11. (Bonus) How Do I Actually Manage a Large Project?

12. Addendum: How do I make a number of torches open a door when lit and make them burn out after a period of time if I do not do light them fast enough?

13. Addendum: How do I change were link lands when he falls in a hole to a lower level?

Question: How Do I Edit “Link’s Graphics” In Hyrule Magic?

One of the latest additions to Hyrule Magic is a “Link’s Graphics” tab which gives you access to the 4bpp image data associated with our hero. It’s a bit tricky to use properly, though.

Exporting Link's Graphics. Palette is Somewhat Messed Up, Since We're Using Wine. But It Should Work Ok.

Exporting Link's Graphics. Palette is Somewhat Messed Up, Since We're Using Wine. But It Should Still Work Ok.

The poses on the left are Link’s animation graphics, and the squares on the right are potential palettes. (You can click on various squares to mess with Link’s graphics, but make sure you click on the lower-left most square before doing any serious work.) To export graphics click the “copy” button, and then go to your favorite image editor and choose “paste”.

You might want to do this in Windows for two reasons. First, although Link’s palette (the bottom row) is shown properly in Wine, all other palette colors are messed up. Although the graphics editor doesn’t directly handle palette data, I personally wouldn’t risk having Wine corrupt your project; Hyrule Magic does enough corruption on its own. Second, one of the best free raster editing programs for Windows has only spotty success under Wine. On Linux, I’ve yet to find a native equivalent —I consider The Gimp to be terrible for pixel-based editing. Either way, you’re on your own for how you edit the copied graphics. Just make sure you save your file as a 4-bit (16-color) BMP file. If you’re using an editor that doesn’t support 4bit BMPs, just save it as a True Color BMP and then convert it later (see the next question).

Let’s give Link a diving-style helmet when he swims. Simply edit the three side-facing and two forward-facing heads with water around them. Give Link the helmet, and also remove the “mouth opening” graphics, like so:

Some of Link's Graphics, Edited to Add a Helmet. Background Faded For Emphasis

Some of Link's Graphics, Edited to Add a Helmet. (Background Faded For Emphasis.)

Make sure that when you edit these graphics, you don’t use any new colors. Only use colors that are already in Link’s graphics. When you’re done, perform a “Select All” and then a “copy” from your image editor, then go into Hyrule Magic and choose “Paste” from the “Link’s Graphics” window. Save your game, and try it out.

Link Is Swimming With His New Safety Helmet On.

Link Is Swimming With His New Safety Helmet On.

It’s not perfect (several more graphics need to be changed to make the animation accurate), but it shows the concept.

If Link’s colors are messed up, or if you got the following error when you pasted in Link’s graphics, then you probably failed to save the image in proper 16-color format. Please see the following question for information about this.

The Error You Get When You Try to Import a Picture With Too Many Colors.

The Error You Get When You Try to Import a Picture With Too Many Colors.

Question: How Do I Save An Image in 16-Color Format?

Hyrule Magic is very picky about the format of its images. Personally, I think it should have exported and imported files instead of clipboard data, but whatever their rationale, the reality is that you often get a “Bad error” message or screwed up colors when you re-import your graphics. The best way to avoid this problem is at the very beginning: by saving the copied graphics properly as a 4bpp, 16-color bitmap before you perform any editing at all.

We’re going to use the free version of Graphics Gale, an excellent raster editing program. First, open the program. Then, choose “File->New”, and enter 128 for the width, 448 for the height, and 4-bit (16 color) for the bit depth. Choose “Ok”.

Now, open a clean Zelda3 ROM in Hyrule Magic, go to the “Link’s graphics” window, and select “Copy”. Then, go back to Graphics Gale. Click on the down-pointing arrow in the palette window, circled below. Then, choose “Load Palette”.

Graphics Gale Hides Its Menus In Non-Standard Context Buttons.

Graphics Gale Hides Its Menus In Non-Standard Context Buttons.

This will bring up a new menu; here, click on “File” and then choose “Import From Clipboard”. You’ll see the palette from Link’s graphics side-by-side with your current palette.

Copying Link's Palette In Graphics Gale.

Copying Link's Palette In Graphics Gale.

Since we want to use all the new (left-hand-side) colors, click and drag to select all of them, and then click once on the first color of the right-hand palette to paste them in order. (You might also try the “All” button, though I’ve never used it.) Then, click Ok. Now, you can simply paste Link’s graphics into the main drawing window (Ctrl+V) and then save the file. At this point, you should be able to open this file using another program; most good editors won’t change the palette without asking you. If that doesn’t work, you can always edit your sprites in Graphics Gale; it’s a decent stand-alone editor.

I’ve heard that Graphics Gale works on Linux under Wine, at least well enough to perform the steps listed above. If you prefer to use another editor on any OS, here’s some instructions on how to save in 16-color format, in The Gimp, Photoshop, and other popular programs.

Remember, though, that saving in 16-color format won’t fix the error; you also have to copy it  in 16-color format. So you should always copy from Graphics Gale before pasting into Hyrule Magic. This is what makes Hyrule Magic difficult to use at times.

Question: How Do I Edit Other Graphics? How Do I Use Zcompress and YY-CHR?

Link’s graphics are unique in that they are encoded with 4 bits-per-pixel (a format supported by most major image editors) and they are (I believe) uncompressed. The remaining graphics are much harder to edit. You basically have to follow three steps:

  1. Decompress the graphics from your ROM to a .bin file.
  2. Edit the graphics using YY-CHR.
  3. Recompress the graphics in your .bin file and insert them into the ROM.

This method is so general that it works for almost any game, so long as you have the right decompression tool. Lunar Compress can handle several popular formats. The only other thing you need to remember is how many bits-per-pixel each graphic is encoded in; please see the Perfect Guide for advice on this, or just look up the palette in Hyrule Magic.

All right, let’s edit Link’s shield graphics. First, open up a command prompt (Start->Run, type “cmd.exe” and hit Enter. On Vista just click Start, type “cmd” and hit Enter. On Linux… you should know how to do this. :D ) Browse to the directory you saved Zcompress to. Make sure you have also saved a clean Zelda3.smc ROM file there as well. Now, type:

zcompress.exe 0 Zelda3.smc Zelda3Gfx.bin

…on Linux, you’ll have to do:

wine zcompress.exe 0 Zelda3.smc Zelda3Gfx.bin

Now, open YY-CHR and select “File->Open”, then browse to your Zcompress directory. In the “File Name” box, type the following and press Enter:

*.bin

This allows you to see .bin files. Now, select your Zelda3Gfx.bin file. The window will load with apparent garbage. You can fix this by clicking on the “Format List” (currently, it shows “4BPP MSX/Genesis) and selecting “4BPP SNES”. Hit PgDown or PgUp to browse through the tileset; you’ll notice that the shapes are familiar but the colors are all wrong.

We've Found Mushroom Boy's Ghost!

We've Found Mushroom Boy's Ghost!

It’s perfectly fine to edit the sprites using these colors, but you can make them much more accurate with just a little extra effort. The following is taken directly from the Perfect Guide:

  1. Decide which sprite you want to edit, and locate that sprite in-game. We’re going to edit Link’s shield, so play through the game in Zsnes until Link’s uncle gives you his shield. Now, save a state (F2) and note the state’s id.
  2. Close Zsnes, and locate the file that was just saved. For example, if you see “State 0 saved”, you will be looking for Zelda3.zst; if it’s “State 7 changed”, you’ll need Zelda3.zs7. Copy this file into the Zcompress directory, to make things easier.
  3. Now, click on “Palette” then “Emulator State Load”. Browse to the Zcompress directory and load your .zs* file. (You might not be able to see it; just type *.zs* into the file window, similar to how you located the .bin file.)
  4. Scroll to your sprite in YY-CHR (the shield is roughly 85% of the way down the screen). Now, click on the palette arrows (circled in red, below) to scroll through all possible palettes. You might not find the exact palette; just find one that’s close.
Link's Shield, With A Slightly-Distorted Palette.

Link's Shield, With A Slightly-Distorted Palette.

Now, edit the shield. Make it bigger. As you do this, make sure that you only use the colors already in the palette. Stealing from colors further along is a bad idea, as it makes it much more difficult to edit those other palettes later. Just make something simple, then choose “save”.

After you’ve saved, close YY-CHR; our Zelda3Gfx.bin file now contains our new shield graphics. We just have to re-insert them. Switch back to your command prompt (in the Zcompress directory) and type in:

zcompress 1 87200 Zelda3.smc Zelda3Gfx.bin

This will re-insert all of your graphics (including the shield) –it will start doing so at 0×87200, which is where a normal Zelda3 ROM expects to see graphics data. Zcompress has the ability to export all graphics as separate files; in this case, you will have to figure out their insertion addresses yourself. But if you always use one big .bin file, the address should always be 87200.

Open your ROM, and play until you get the shield. Don’t your new graphics look nice?

We've Added a Blue Face To Link's Shield, and Enlarged It.

We've Added a Blue Face To Link's Shield, and Enlarged It.

Of course, we can do better than that! Go into the palette editor in Hyrule Magic and scroll down to “Shield 0”. Change the dark blue to white, the light blue to navy, and the white to red. Now, re-arrange your graphics in YY-CHR to look spookier. Ooh~~~ skull shield!

We've Changed the Face Into a Skull! Note That Link's Uncle Uses the Same Shield As His Nephew.

We've Changed the Face Into a Skull! Note That Link's Uncle Uses the Same Shield As His Nephew.

Are we having fun yet? Keep in mind that some graphics are more stubborn than others. I still can’t manage to figure out how to change the treasure chest graphics. Also, some tiles are mirrored, so if you’re editing, say, tombstones, you don’t have to bother looking for the “right half” graphics —they’re simply mirrors of the “left half” graphics.

Question: How Do I Edit Overworld Tiles? What’s This 16×16 Tile Nonsense?

As mentioned previously, Zelda 3 is hyper-compressed. This makes overworld editing slightly more complex than it should be. Zelda3 enforces a three-tier hierarchy on overworld graphics, the primary purpose of which is to support complex dirt paths, mountain cliffsides, and rivers —all without wasting tiles. It’s pretty natural once you get used to it.

Each world-map “tile” is 32 pixels square. This is just a convenience measure; you cannot, for example, assign properties to any of these tiles. Rather, you assign properties to the four 16×16 tiles that make up one world map tile. 16 pixels square is the “logical” tile size in Zelda. (For now, we ignore “big” tiles like the 32×32 rocks Link can pick up.) Each of these, however, is made up four 8×8 tiles which act as a “palette” to construct the larger tile from. If you have, say, a sign which is symmetrical about the Y-axis, it makes sense to simply draw the left-hand-side, and then mirror that tile on the right-hand side. That’s what 8×8 tiles are used for.

We’re tired of drawing our own graphics, so let’s just re-use existing tiles to make something new. We’ll put the top of a bush onto the bottom of a rock, thus making a sort of “chia pet” tile. This tile is a rock, but it’s also easy to pick up; you don’t need any special gloves. We’ll put this tile outside of Link’s house.

That’s our goal; now let’s do it. Open a clean Zelda3 ROM in Hyrule Magic and go to overworld area 2A. Now click on the “movement” arrows (in red, below) to scroll around until you find Link’s House. Hint: Just move once to the right.

Navigating Is Done With the Four Arrows Circled Here.

Navigating Is Done With the Four Arrows Circled Here.

Remember the room ID (2C) and you can just open it directly next time, or use the “jump to” button. Now, make sure the “Select” button in the top toolbar is depressed, and click/drag to select the two rocks in front of Link’s house. Then, right-click on this square to zoom to the “palette” tile on the right. (You actually don’t have to left-click drag, but I like to have the same tile highlighted on the left and right halves of the screen.) You’ll end up with the following:

Left-Click the Left and Then Right-Click the Left to Load the Right. Easy.

Left-Click the Left and Then Right-Click the Left to Load the Right. Easy.

Double-click on the palette tile (right-hand side) and it will open in the 32×32 block editor. You can see that it is composed of four 16×16 tiles: two rocks, and two “jump down” cliff tiles.

The 32x32 Tile Editor.

The 32x32 Tile Editor.

Right-click on each tile to see its palette tile load in the right-hand side of the 32×32 block editor. Now, scroll down through the various 16×16 palette tiles until you find the dark-colored rock tile. You need to be careful when making new tile graphics; some of them are reserved, some cannot appear on every overworld screen, and some have hard-coded properties. So, we’ll just edit an existing tile that we know is safe: the Lv. 2 rock tile. Double-click on this tile to open it in the 16×16 block editor:

The 16x16 Tile Editor Requires Some Finesse With Palettes.

The 16x16 Tile Editor Requires Some Finesse With Palettes.

On the left is your tile, composed of four 8×8 tiles. On the right are your “palette” 8×8 tiles. Unfortunately, Hyrule Magic cannot know which palette your map will be using, so the colors are a bit screwy. Try typing various numbers into the “palette” dialogue until the graphics normalize. Palette number “5” or “6” is a good choice. If you double-clicked on an 8×8 tile, you could edit it further, but we want to just build our new tile from the existing “bush” and “rock” graphics.

However, there are several different bush/rock graphics, so which one should we copy? Close the 16×16 tile editor and right-click on the original (Lv. 1) rock. Then, double-click on its palette entry. Finally, right-click on each of the bottom two tiles. They are both palette 5, block 434, and one of them has the “X flip” attribute set. Cancel, and do the same for a “bush” tile from the world map. You’ll see that the top two tiles are both palette 6, block 384, and one of them has the X-flip attribute set. Now that we know this information, we can easily construct our new tile:

  1. Re-open the “black rock” tile in the 16×16 block editor.
  2. Enter “5” for the palette.
  3. Type “434” into the “Block” window. Left-click on the lower-left tile to place this.
  4. Click the “X flip” check box. Left-click on the lower-right tile to place this.
  5. Uncheck “X flip” and enter “384” for the block and “6” for the palette. Left-click on the upper-left tile to place it.
  6. Check the “X flip” box and left-click on the upper-right tile to place it.
  7. Change the “Block type” to “39”, the same type as bushes, so that you can pick it up and slash it with your sword.

You should now have a proper bush-rock-combination tile. Click “Ok” and then left-click on your new tile in the palette window, then left-click on each of the two rocks to change them to chia tiles.

We've Successfully Merged Rocks and Bushes.

We've Successfully Merged Rocks and Bushes.

Then, click “Ok”. The bushes in front of Link’s house should now appear as rock-bushes. It is possible, however, that your new tile is all black. You’ll first notice this in the 16×16 block editor:

Sometimes The 16x16 Palettes Won't Refresh.

Sometimes The 16x16 Palettes Won't Refresh.

If this is the case, simply select “Ok”, and then look for the “Palette” box in the overworld map window. Currently, the entry is “0”. Change it to “5”, then change it back to “0” to fix the problem.

You will probably also get a warning message when you save the game:

Editing the World Map Can Lead To Difficulty Saving.

Editing the World Map Can Lead To Difficulty Saving.

This is because several areas share some common tiles (like the dark black rock you edited). Personally, I’m wary of selecting “Yes”, since I don’t trust the editor to “fix” this problem. Clicking “No” has one obvious side-effect, though: it replaces all black rocks with your rock-bushes. I find this acceptable; we’re only testing out the area near Link’s house anyway.

Now, simply play the game, and enjoy your new bush-rocks.

The Bush-Rocks Work... Mostly.

The Bush-Rocks Work... Mostly.

There are a few minor issues, such as the fact that these new tiles occasionally drop bombs, or the incorrect “full rock” graphic that appears when you pick it up, but the general effect is nice.

Lunch~

It’s lunch time and we’re not even halfway done. Grab a sandwich (I recommend a Reuben) and get ready for more barrels of fun when we return.

Question: How Do I Change Flute Locations?

Changing the 8 flute locations (we ignore the special 9th location) is quite easy to do, but it requires two rather unrelated steps:

  1. Change the location of the flashing number on the world map.
  2. Change the location the bird drops you off at on the proper Overworld screen.

Neither step is difficult. First, open Hyrule Magic and click on “World Maps” and then “Normal World”. Change the selected marker from “Hyrule Castle” to “Flute Locations” —the very last item in the drop-down list.

Eight of the Nine Flute Locations, As Seen On The In-Game Map.

Eight of the Nine Flute Locations, As Seen On The In-Game Map.

Personally, I’ve always found location 8 to be a bit lame. Let’s move it to a more useful place: the Great Fairy’s island. To do this, click on the “Move” button in the top toolbar, then simply drag the pink 8 to the center of the lake.

Drag the Flute Location to the Island.

Drag the Flute Location to the Island.

Save, and close the world map editor. If you opened the saved game now, you’d see the flashing number 8 in its new location, but the bird would still bring you to the old location. To fix this, open “Overworld” area 3F. Click on the “Transport” button to show the whirlpools and bird locations, and then right-click on “Fly-8” and choose “Remove”.

Hyrule Magic Will Complain If You Don't Remove the Old Flute Point First.

Hyrule Magic Will Complain If You Don't Remove the Old Flute Point First.

Now, move to area 35 and right-click on the island, then choose “Insert bird location”. Congratulations! You’ve moved the drop-off point.

Adding a New Flute Location is Easy Once You Know Where to Look For It.

Adding a New Flute Location is Easy Once You Know Where to Look For It.

Save your map. Before testing it out, you should probably short-circuit a few of the doors in the castle, to make evacuating Zelda easier. Also, put the Flute in the treasure chest in Link’s house. This way, you can start a new game, rescue Zelda, and activate the bird in Kakariko Village without wasting too much time. The end result is exactly what we expected, except…

The Map Is Properly Up-To-Date.

The Map Is Properly Up-To-Date.

Link Also Zooms In To the New Location Correctly.

Link Also Zooms In To the New Location Correctly.

Our rock-bush tile is still in effect from the pre-lunch hack. This has some unintended consequences:

Did Anyone Else Notice That Those Ice Men Were Super-Damaging?

Did Anyone Else Notice That Those Ice Men Were Super-Damaging?

But that’s nothing you can’t handle, right? ;)

Question: How Do I Edit The Game’s 3-D Graphics? How Complex Can These Graphics Be?

The answer to the second question is: not very. The combination of points and faces is limited to 104 units in total, but I’m not sure how these are tallied. (My personal guess? 3 units for each point you place, and one unit for each point in a face.) Also, the triforce segments and crystal shards share the same total point set. For now, you’ll just have to experiment with the editor to see what its limits are.

To answer the first question, open a clean Zelda3 ROM in Hyrule Magic and click the “3D Objects” tab. Then click once on the black, gridded area to focus it. Now, you can rotate the scene (arrow keys) and zoom (num pad plus/minus). The current view, of the crystals, is somewhat difficult to grasp, so hit the right arrow in the upper-left of the screen to switch to editing the triforce shards.

Hyrule Magic's 3D Shape Editor is Impressive, But Slightly Tricky to Use.

Hyrule Magic's 3D Shape Editor is Impressive, But Slightly Tricky to Use.

You should then click on each point on the top surface of the triforce to see its location in 3D-space, which will appear in the upper-left corner of the screen.

Click On A Point To Show Its Position In 3D Space.

Click On A Point To Show Its Position In 3D Space.

Here, we can see that this point is located at X:A8, Y:58, Z:76. Click on the remaining points and jot them down. These coordinates, of course, are hex values.

Face X Y Z
Top Face A8 58 76
58 58 76
80 A8 76
Bottom Face A8 58 8A
58 58 8A
80 8A 8A

The triangle is 0×14 points thick, with the bottom face flat on 0×8A and the top face flat on 0×76. The remaining Cartesian co-ordinates are distributed as an equilateral triangle:

Our Equilateral Triforce Piece.

Our Equilateral Triforce Piece.

We would like to replace the triangle with a nice hexagonal shape. However, if you try to add a point, you’ll notice that we don’t have any more points to work with. So, click the left arrow (at the top-left of the window) to switch back to the crystals, and then click on any one of the points. Hit “Backspace” to delete this point —the cursor will then automatically jump to the next point. Continue hitting “backspace” until all points have been deleted. Then, hit the “right” arrow to go to the triforce triangle, and delete all points here too.

Now, draw a rough sketch of your hexagon; it should be roughly the same size as the triangle. Drawing a sketch is important, as it allows you to identify vertices easily. You should end up with something like this:

Our Hexa-Force is At Least Twice As Powerful As the Triforce!

Our Hexa-Force is At Least Twice As Powerful As the Triforce!

Now we have our six points in a single z-plane:

(6C,58), (94,58), (A8,80), (94,A8), (6C,A8), (58,80)

You’ll notice that I wrote these out in a counter-clockwise order. The reason will be important later: Hyrule Magic creates an “outward facing” shape when points are connected counter-clockwise.

Now, we need to enter these points. To enter the first point, (6C,58), do the following:

  1. Click the “Add Points” button, and click once on the black area. This will pull up a blue line and two green lines, indicating the temporary placement of your point.
  2. Click on one of the green lines to place your point. Place it anywhere; we’ll move it manually to make sure it ends up at the exact position we choose.
  3. Click the “Move” button, then left-click once on your new point. Now, press the A or Z key until the point’s “Z coordinate” is at 76. Then, use the numeric keypad (this part sucks on a laptop) to shift the point’s X/Y coordinates to 6C and 58, respectively.
Our First Point, Positioned Correctly In 3D Space.

Our First Point, Positioned Correctly In 3D Space.

Continue to add all the points; the order you add them in is not important (only the order you connect them in). The Z-component will always be 76, since we’re only building the top face for now. Then, after all points have been added, rotate the view to the origin (the easy way: save, close, and re-open your ROM in Hyrule Magic), click the “Add Faces” button and click on each point in order. It doesn’t matter where you start, so long as you go in a counter-clockwise direction, and end at your starting point. As you click on each point, a blue line will trace the current shape you’re creating. You should have something like this when you’re done:

Our First Hexagon Surface. 1-Dimensional.

Our First Hexagon Surface. 1-Dimensional.

Save your file and close the editor. Start up the ROM, and you’ll see some blinking hexagons:

Our 1-D Hexagons Often Vanish, But Prove the Concept.

Our 1-D Hexagons Often Vanish, But Prove the Concept.

This is fine; the back face of the hexagons is invisible; we’ll fix that later. If, instead of loading, your game shows some glitchy graphics and then freezes up, you probably edited the crystal’s graphics instead of the triforce’s graphics (d’oh!). Just delete all points, hit the “right” arrow, and start again.

Let’s add the second set of points. Load your ROM in Hyrule Magic and go to the 3D Objects editor, pressing “right” to go to the triforce graphics. You should see the hexagon facing you directly; hold “Down” to rotate the screen until you are seeing the back of the hexagon. Now, add six new points to the object. They have the same X and Y coordinates as the previous points, but their Z-coordinates are all the same: 8A.

After you’ve added these points, hit “Up” two or three times so that you can tell these points apart from the first set. Then, connect all six in a counter-clockwise direction using the “Add Face” button. You now should have two hexagons facing away from each other:

Our 2 Faces of the Coin Have Been Added. One Should Always Be Invisible.

Our 2 Faces of the Coin Have Been Added. One Should Always Be Invisible.

There’s a small chance that you’ll corrupt your first face by adding the points for the second. In this case, the “Delete Face” button might not even work! You’ll have to delete one of the points contained in the face. (In my case, there was a weird triangular shape; I just deleted and re-added one of the points of that triangle.) In general, you’ll want to add all points before you connect any faces.

Either way, you only have one more task before we’re done: just add each of the six rectangles that connect the top and bottom two hexagons. Rotate the window as you go, and make sure you connect all points in a counter-clockwise fashion. When you’re done, you’ll have a nice, coin-ish object.

My Favorite Picture From This Entire Post: Your Complete Coin.

My Favorite Picture From This Entire Post: Your Complete Coin.

Save and exit, then load your game. My only regret is that I didn’t actually do the math properly, so our coin is a bit “longer” than it should be.

Our Coins Rotate Into Position. Hooray, 3-D Editor!

Our Coins Rotate Into Position. Hooray, 3-D Editor!

Does anyone else find it interesting that bsnes and zsnes display the opening zoom-in differently? Also, although it should be painfully obvious by now, you can change the palette used for the triforce just like anything else in the game:

Go Rainbow Laser Triforce!

Go Rainbow Laser Triforce!

Question: How Do I Change The Music For An Area?

I’m pitching a changeup with this one. Sorry, but I need an easy section for once. Changing the music for an area is just a matter of tracking down the right dialog box. The one you want is in the “Overworld” editor, called “Properties”.

Here's The Overworld Properties Dialogue.

Here's The Overworld Properties Dialogue.

Since I’m punting on this one, I’ll at least explain this box properly. Besides a section that lets you change the sign text (by ID), all eight options relate to music and sound. The four categories relate to the four stages of the game:

  1. “Beginning” — From the game’s beginning until you get Zelda to the sanctuary.
  2. “After rescuing Zelda” — From the sanctuary until you retrieve the Master Sword (regardless of if you’ve visited the Dark World yet).
  3. “After getting the Master Sword” — From the time you get the Master Sword (Lv. 2) until the time you get teleported to the Dark World from Hyrule Castle.
  4. “After beating Agahnim” — From the time you defeat Agahnim and are teleported until you beat the game.

The “music” option is simple: What music plays for this section of the screen during the designated time period? Try changing all “music” to “Forest” to see what happens. You should be able to guess. The “ambient sound” setting loops a background noise constantly when on. Try changing the sound to “Light Rain” for all four time slots. Now, when you enter this particular screen, you will always hear rain and the music from the Lost Woods. In the Dark World, there is generally only one time slice available. Try mixing and matching various sounds and music tracks to get the effect you want; actually changing the music itself will be covered in an “Advanced” tutorial later.

Thanks for the easy question~

Question: How Do I Make An Enemy Drop a Key? How Do I Choose Which Enemy Drops the Key?

This is a reasonable question to ask; although long-time Zelda players know that certain enemies drop keys in certain rooms, the Perfect Guide says to just “put the key at 00,3C,2” and pretty much leaves it at that. The actual method is slightly more involved:

  1. Open a clean ROM in Hyrule Magic. Go to Entrance 04, the main entrance to Hyrule Castle.
  2. Click on the “Sprite” checkbox, and then right-click anywhere on the map and choose “Insert An Enemy”. Scroll through the list until you find the “Key” entry and choose Ok. Then, click and drag that sprite to location X:00, Y:3C. Now, press the hyphen (minus) key to make it jump to “BG2”. Finally, press the period key several times until its property (P) value is 18.
  3. Now, press the left arrow key several times, to move to the first sprite in the room. Hit the right arrow key to scroll through the sprites one-by-one, and pay attention to the order of the sprites. Your key sprite will appear directly after the enemy that drops it.
  4. So, click on your key sprite and press “V” or “B” to move it forward or backwards in the order until it is directly after the enemy you want to drop the key. You might have to use the left/right arrow keys to check, in case you lose track of the current order.

Make sense? Here’s a picture of my screen after I’ve edited the key sprite and positioned it properly. Note the properties tab (circled) and the order of the sprites (drawn as a red connected path). According to this screenshot, you can determine that the key will appear after killing the “Green Soldier” enemy.

Annotated Screenshot Showing How Enemy Order Determines Who Drops the Key. Also Note the Object's Properties.

Annotated Screenshot Showing How Enemy Order Determines Who Drops the Key. Also Note the Object's Properties.

Question: How Do I Add a Nifty Logo To The Title Screen?

This question’s more a matter of patience than anything. The “Menu screens” tab in Hyrule Magic lists a series of single screens that you’ll recognize from the game. For now, open the last menu: “Destination List”. The player sees this menu whenever he starts the game any time after rescuing Zelda. There are several problems with the initial view, however.

The Destination Picker Window.

The Destination Picker Window.

For one, the menu box and the fairy used to select your location are not visible, making it very difficult to decide where to place the logo. This occurs because such graphics are sprites, whereas the background is just a static image. We’ll get around this problem by using placeholders.

The next issue is that we can’t even see the palette tiles that make up the text! Click on the BG3 radio button; ah, there they are.

An altogether different sort of problem is one you should be used to by now: the palette’s miscolored. Keep trying numbers until you get something that looks good; I used 6.

However, we’ve now reached the main issue: editing the last five menu items usually doesn’t work. This includes the “Destination List”, of course. I’ve tried adding tiles on BG’s 1, 2, and 3, with or without the “In Front” property specified, all to no avail. There’s a chance that changing the palette would fix the problem, but I’d had enough by then. Close the “Destination list”; we’ll have to edit something else.

Double-click to open the “Title Screen”, which is much easier to edit. The only thing I haven’t been able to add yet is BG3 sprites. (Actually, I know they’re in there, since the Master Sword is obstructed by their shadows. But I can’t get them to show up.) This isn’t such a big restriction, since we’ll be drawing our own graphics anyway.

Menu screens are somewhat like dungeon screens, which makes it surprising that the same editor logic isn’t used. Instead, we’ve got a bunch of random keys which help us accomplish our tasks. Make sure the “Move” button at the top of the window is pressed down, and click once on the blue sky above the word “Zelda”. Notice how several tiles were selected? This is a common theme in menu screens: a “block” of editable tiles. You can drag a block of tiles with the mouse, and increase or decrease its size with the period and comma keys. Pressing m changes if “size” refers to width or height. If you want all tiles in a block to have the same value, press the hyphen (subtraction) key to switch between normal mode and “fill” mode. Note that dragging a block to the edges of the screen can sometimes cause it to auto-fill the entire width, just like in the Dungeon Editor.

Before we place our logo (which will be 3×2 tiles), we first have to place a dummy sprite and make sure it shows up in the right place. We’ll use two blocks of 3 tiles. Switch to “Move” mode, and switch to “BG1”. (Note: Click the radio button BG1, not the check box BG1. The check box just toggles the visibility, to make editing easier.) Now, right-click somewhere below the bottom of Hyrule Lake. A new block should appear. Hit the period key twice to make it expand.

Right-Click With "Move" Selected To Add A New Block. Then Resize It.

Right-Click With "Move" Selected To Add A New Block. Then Resize It.

Now, switch to “draw” mode, and pick any colorful tile. Click on it in the palette on the right, then left-click and drag over your new block to fill all three tiles in that block with your new tile.

Left-Click and Drag to Color the New Block. Hit Detection Is Sometimes Dodgy Here...

Left-Click and Drag to Color the New Block. Hit Detection Is Sometimes Dodgy Here...

Switch back to “Move” mode, and right-click one tile below the block you just entered. Press period twice to expand it to 3 tiles. Switch to draw mode and color it with another random tile from the palette. Now, switch back to draw mode and drag both of your new blocks until they’re in a sensible position.

Our First Tentative Placement of Logo Stand-In Graphics.

Our First Tentative Placement of Logo Stand-In Graphics.

Now, save your game, and load it in Zsnes. Your faux-logo should be in the right spot; if not, go back to Hyrule Magic and move it until it shows up properly. In particular, if you can’t see it at all, something is overlapping it. Or, you’ve chosen a bad palette. Try re-drawing your title screen graphics after first changing the “palette” in the Menu editor. I used palette 4 pretty effectively.

After Some Fiddling and Re-Coloring, Here's Our Logo Placeholder Graphics in Zsnes.

After Some Fiddling and Re-Coloring, Here's Our Logo Placeholder Graphics in Zsnes.

Now that we’ve got the logo in place, we just have to edit its graphics. You should know how to save graphics in 16-color format (see question 2), and you should know how to edit Link’s graphics by exporting them (question 1) or with YY-CHR (question 3). Either of these will work. First, go back to your menu editor, and select the proper palette. Then, double-click on one of the tiles in the right-hand box that you previously added. A new window should pop up.

The Import/Export Window for Menu Tiles.

The Import/Export Window for Menu Tiles.

You should know what to do here: copy and paste these graphics into Graphics Gale and save it in 16-color format. Then, open Paint.NET and edit some of the tiles (using only palette colors) to create a desirable logo. Make sure you don’t edit any of the tiles that appear in the title screen. Some of the all-black tiles are probably safe to edit, but if you don’t mind messing up existing graphics in other menus, just edit tiles that you know for sure don’t appear in the title screen. Finally, copy/paste your edited spritesheet from Graphics Gale back into Hyrule Magic, and then switch to “Draw” mode and click on each of your six new tiles to insert them where your placeholder graphics used to be. Congratulations, you’ve made a new logo!

Our Completed Logo. Note: The Graphical Glitches Are Unrelated To The Logo; I Just Messed Up The Import.

Our Completed Logo. Note: The Graphical Glitches Are Unrelated To The Logo; I Just Messed Up The Import.

Of course, if you’re willing to track down and manage the palette colors yourself, you can probably make a much nicer logo. We won’t cover changing the palette on image imports, as the Perfect Guide has a few nice examples on that.

Bonus: How Do I Actually Manage a Large Project?

Instead of giving you homework, I’ll give you my own personal bit of advice on how to manage a real project with Hyrule Magic. Learning the program is fairly easy, and coming up with creative dungeon ideas is either simple or impossible —it’s more of an art than a science. So the only real challenge is the organizational aspect. What if you want to steal a palette color from Link’s shield to use in his hat? Even if you set aside the same color in all three “shield” palettes, you’ll still have to jot this information down somewhere. What if you want to play around with the enemy palettes available in each dungeon, to make, say, crystal switches available everywhere? Hyrule Magic’s occasional corrupting of your ROM doesn’t make this any easier.

The key idea here is that you should somehow extract a “source” version of Zelda3, and then re-insert it into the ROM when you want to open it up in Hyrule Magic. The ZCompress tutorial I wrote is a good example of this. Some programs go even further, re-writing the game’s ASM to force you to change the ROM. Lunar Magic sometimes does this, relocating pointers and the like. However, I see no need to toss out Hyrule Magic for, say, Open Zelda. Can’t we find a way to perform our extract-edit-insert cycle without mangling the ROM so badly that Hyrule Magic can’t open it? That way, we can try out new ideas (and do dungeon design, etc.) in Hyrule Magic, and take advantage of all the work done by its developers on understanding the Zelda3 format. On a related note, can’t we add some capability for backup?

I forsee two possible solutions:

  • Use a makefile to “build” your ROM from a series of text and binary files. As you decide to edit a new part of Zelda3, you can create a text file representing that particular aspect in a useful way. Then, you can define logic for importing it into the ROM. You’ll also need some way to “extract” dungeon rooms from Zelda3, and save them as .bin files (just the HEX values, really). That way, you can back up your work as you go. If Hyrule Magic corrupts your data, just delete the ROM and run a “make” command to generate a new one. This approach also allows you to throw your project into a repository, making it easier for others to help out with.
  • Use Microsoft Excel and VBA macros. Although there are several great programming languages out there, Excel is one of the few good 2-dimensional ones. Consider: you write a macro to import a PNG of Link’s poses and display each one in a separate cell. Then, you generate cells representing each palette color. Another sheet can contain information on Zelda3’s palettes and how they relate. Your code can then check and tell you if you have any free palette colors available, or if you’ve overrun the limit. Linking this up with a program like Auto Hot Key can give you primitive control over importing data back into Zelda3.

Personally, I’d try to combine the two. Try to define text/image/binary formats for each of Zelda3’s various data. When possible, keep things “native” (e.g., PNGs) and use a common naming convention (e.g., “Link_4bpp.png”, “Weapons_3bpp.png”) to allow your tools to enforce rules about, say, bit depth, that are hard to enforce using general graphical formats. Then, make Excel macros to import this data and generate whatever views you need. Perhaps, for example, you actually prefer editing maps using Excel. So make a map editor tab. Then, use a Makefile to generate a Zelda3 ROM from your text/binary formats. This allows you to backup into an SVN repository. Moreover, it allows you to play around editing things in Hyrule Magic. I imagine Hyrule Magic will be faster to experiment in than anything you could come up with in Excel. So leverage that. Each tool has a use. To me, Hyrule Magic’s use is more geared towards general editing of Zelda3 than managing a new Zelda3 project. But if you’re content fiddling with it for all eternity, then by all means just skip this section —and make constant backups!

Happy Hacking!

Addendum

A few questions were asked after the initial post. They were pretty reasonable, so I pecked away at them for a few days and came up with some answers.

Addendum: Question: How do I make a number of torches open a door when lit and make them burn out after a period of time if I do not do light them fast enough?

First of all, have a look at this passage from the Perfect Guide:

$04F0-$04FF Timers for torches. Starts at #FF and counts down to 0. Setting it before the torch is lit is a bad idea. It will not cause a torch to light, nor will it brighten the room. Note that this range probably indicates we can have up to 16 torches in an area.

This is in the section on “RAM Data”, and it has some interesting implications:

  1. Torches probably burn out on their own.
  2. Apparently we could (with some ASM hacking) make a puzzle in which several torches burn out after different amounts of time. That would be a fun puzzle.
  3. Seems that we can’t have more than 16 torches in one area/puzzle. Be careful! Multiple rooms might still be considered the same “area”; you’ll have to test this.

The Guide’s content for torches is a bit sparse (as always), so let’s look at a real example from the game. It’s been a while since I played Zelda3, so the only example I remember is that traumatic moving platform puzzle in Turtle Rock. Open up a clean ROM in Hyrule Magic, and browse to Entrance 35. Then, move up and then right until you get to the puzzle room.

This Puzzle Occasionally Appears In My Nightmares

This Puzzle Occasionally Appears In My Nightmares

Click on the “Torch” button in the lower-right, and then right-click on the map. It seems you can add and delete torches, but there’s no special property stating what these torches do.

Click on the “More” button in the top panel to view the map’s extended properties. Notice the tag “Light torches to open”. Click “cancel.

Note the Entry for "Tag 1"

Note the Entry for "Tag 1"

Click on the “3” in the “Edit” box and then click once on the door at the top of the screen. You will see that its “Type” is set to “12”. Maybe Type 12 is a door that opens when all torches are lit? Let’s try it out.

Close the Turtle Rock dungeon window and click on Entrance 7D. Now, click on the “Torch” option, left-click on the existing torch, and drag it closer to the upper-middle-left door. Save.

Add a Torch in the Upper-Left Corner

Add a Torch in the Upper-Left Corner

Now, in the “Edit” box, click on layer “3”, and then left-click once on the door near the torch. Now press “M” several times until the “type” changes to 12.

The Door Is Now Locked

The Door Is Now Locked

Don’t forget to set Tag 1 to “Light torches to open”. Otherwise, lighting the torches won’t affect anything.

Save the ROM, and try it out….  it doesn’t work. Hmm…. a bit of Googling finds this tidbit:

“In order for light torches to open door/get chest to work, you'll need exactly 4 torches in the room, no more, no less.” (Source)

Good to know. Add three more torches to our room. (Click on “Torch”, then right-click and choose “Insert a torch”.) Make sure you can access all four from the top half of the room. Note that you might have to delete three torches from some other room in the game before Hyrule Magic will allow you to save the game.

Our Three Additional Torches

Our Three Additional Torches

Save, close, and try it out. Yes! It works!

The Door Behaves As Expected

The Door Behaves As Expected

Also note that, as expected, the torches extinguish after a certain amount of time.

Addendum: How do I change were link lands when he falls in a hole to a lower level?

There are two types of holes Link can fall in: “Entrances” from the overworld (like the one we just edited, below the bush in Hyrule Castle) and “Multi-Storey” holes inside dungeons. Well, there’s a third type of hole; one that damages you when you fall into it, but your question wasn’t really about that. Your question is about the second type, but I’ll cover both here.

Let’s start with Overworld hole entrances. First, you need to specify which entrance the hole will lead to. Open your Zelda ROM and click on “Overworld” then Area 1B. This is the main castle area. Now, click on the “Hole” toolbar item, then find the tag “7D – Hole” on the map screen. The “7D” in the hole’s name refers to the entrance it leads to; remember from the previous question that we opened Entrance 7D to edit the room where you find your uncle.

Click once on “7D – Hole” and then use the N and M keys to change which entrance this refers to. Change it so that that tag now reads “09 – Hole”. Then, save the map.

The Secret Garden Entrance, Modified

The Secret Garden Entrance, Modified

If you start Zelda3 and walk into that hole, you’ll end up in the Desert Palace. You will “fall” directly onto the main entrance. If you open up Entrance 09 in the Dungeon tab, go to the “Starting Location” section and change “Y” to, say, 400, then Link will fall in slightly above the entrance.

You Can Now "Fall" In To the Desert Palace.

You Can Now "Fall" In To the Desert Palace.

However, he will also appear in this spot when he walks in from the outside door, which will look weird (try walking out then in). So how can we fix this? Well, we’d need to use two Entrances: one for falling in, and one for walking in. Notice how the Hyrule Castle area uses Entrance 7D for falling in, and Entrance 32 for walking in. They both lead to the same room, but with different “Starting Locations”.

So how about multi-level dungeons? The Perfect Guide has this to say about holes in dungeons, in the “Room Information and Properties” section:

Hole/Warp: Directs the holes or warp tiles. When you fall through a floor or warp to another room it will always be in the exact place in the new room as you left in the old.

To see this in action, open Entrance 33, the Tower of Hera, and click on the “More” button in the top panel.

The Tower's Map Properties Dialog

The Tower's Map Properties Dialog

Notice the “Hole/Warp” box has a value of 167? This is where you land if you fall into the big jars in the top part of the room. Just to spice things up, let’s add a new hole to fall into. Click on the “2” in the “Edit” tab, then right-click near the entrance and choose “Insert an Object”. Browse to object 0A4, and click “Ok”. Then, drag it somewhere noticeable, and save.

Our Main Entrance Is Now Hazardous!

Our Main Entrance Is Now Hazardous!

Click the “Jump” button and type in 167 to go to the target of the hole. Now, insert an object (any object) near the bottom of the screen, on layer 1, so that we can test if you fell into the right room. Save.

Add Anything, Just So We Can Tell If We Landed In the Right Room

Add Anything, Just So We Can Tell If We Landed In the Right Room

Now, go back to overworld area 1B, and change the hole under the bush to lead to Entrance 33. Save, and start the game. Fall into the first hole (near the castle) to warp to the mountain palace…

The Shortest Distance Between Two Points Is a Modified Entrance.

The Shortest Distance Between Two Points Is a Modified Entrance.

…and fall into the second hole to test that you end up at the exact same location in room 167.

We Landed In Our Abandoned Room

We Landed In Our Abandoned Room

So there you have it! Zelda3’s all about room/dungeon/nonsense management, and warp tiles/holes are just another great example of where a little knowledge goes a long way.

June 5, 2009

[RM-3] Speech Balloons That “Type” Their Message

Blog Info: I’m mostly done reading a great 65816 (SNES) assembly guide; this guide is so sensibly written that I think I’m actually ready to do a fun text hack for my favorite game of all time. So this week’s hack will help me prepare by introducing the algorithm in a high-level language. Plus, I think the RPG Maker community will like this one.

Back-story: Ok, our last attempt at speech balloons kind of flopped. They stretched text, they jittered, and they just popped into existence. Their worst sin, in my opinion, is that they didn’t “open” like the default RPG Maker ones, nor could they “type” their text out letter by letter. Let’s fix everything in one fell swoop.

Goal: Spruce up our speech balloon code, particularly in regards to opening/closing them when the player walks within range, and typing their message text out letter-by-letter.

You should read through tutorial RM-2 before starting this task. Instead of using a clean (new) project, I’m going to start where we left off after that tutorial. Don’t worry, I flushed my own project and copied it letter-for-letter from RM-2. One thing to note:

  • Your “Talkbox_Window” module needs to be in a location that the interpreter can find. For example, add it below the “Windows” tab, or “Materials”. If you just add it anywhere (say, at  the end of the file), you might get an error when you try to run the program.

I added this factoid to the original article, too. People, please tell me if you can’t get your code to work following one of my guides; I’ll be happy to fix it.

Step 1: Variable Width and Jitter Reduction
Add the following line to “Main” right after Graphics.freeze:

$Debug = Window_Base.new(544, 0, 544, 416)
$Debug.windowskin = nil

This just gives us an “offscreen” area to calculate stuff on. If you’re combining tutorial RM-1 with RM-2, you can ignore this, and just use the regular $Debug window for offscreen stuff.

Replace the Talkbox_Window class with the following one:

#A slightly improved RM-2 talkbox
class Talkbox_Window < Window_Base
  #RPG Maker defaults
  MAX_WINDOW_WIDTH = 544
  MAX_WINDOW_HEIGHT = 416
  WINDOW_PADDING = 32

  #npc is a GameEvent
  def initialize(npc, txt)
    #Figure out a proper size, and initialize it offscreen
    my_size = $Debug.contents.text_size(txt)
    super(0, 0,
       my_size.width + WINDOW_PADDING,
       my_size.height + WINDOW_PADDING)

    #Put off setting this variable until later
    @offset_y = -1

    #On top and invisible
    self.z = 90
    self.visible = false

    #Draw the text once
    self.contents.draw_text(0, 0, draw_width, draw_height, txt)
  end

  #Call this to update the talkbox’s position over an NPC
  def update_pos(npc)
    #Store the npc’s height if we haven’t done so yet
    @offset_y = npc.sprite_height if @offset_y==-1

    #Update this talkbox’s position
    self.x = npc.screen_x - self.width/2
    self.y = npc.screen_y - self.height - @offset_y
  end

  #Get the width of the client area
  def draw_width
    return self.width - WINDOW_PADDING
  end

  #Get the height of the client area
  def draw_height
    return self.height - WINDOW_PADDING
  end
end

The MAX_WINDOW_WIDTH and MAX_WINDOW_HEIGHT variables just happen to be the window sizes RPG Maker ships with. I couldn’t for the life of me find these anywhere in the documentation, so I just stored my own values here. The DEFAULT_PADDING represents how much space is used as “border space” by any RPG Maker window —if you don’t add this value to each window you create, you may  find that everything you draw  is “cut off”.

We cache the return value of sprite_height because we’re worried it will affect performance. We’ll describe why later.

The remainder of the code is pretty simple to understand, even though not all relevant functions have been implemented yet. We’re basically resizing the talkbox to hold all of its text, and centering it above the NPC in question.

Now we need to add all those missing functions. Open Game_Event and add the following two member definitions:

  def sprite_width
    #Easy if this is a tile-based event
    return 32 if @tile_id > 0

    #Harder if this is a sprite
    unless $scene.is_a?(Scene_Map)
      return 0   #A generic Scene isn’t good enough
    end
    return $scene.get_event_width(self)
  end

  def sprite_height
    #Easy if this is a tile-based event
    return 32 if @tile_id > 0

    #Harder if this is a sprite
    unless $scene.is_a?(Scene_Map)
      return 0   #A generic Scene isn’t good enough
    end
    return $scene.get_event_height(self)
  end

There’s probably a better way to get this information, but this will have to do until I get a chance to read all the RMVX documentation. Basically, we just return the size of a tile or the size of a sprite depending on what this event is represented by. Of course, our caching code will position talkboxes improperly if you keep changing your event’s graphics from a Tile to a Chara representation. You can remove the caching if this bothers you, but I find it to be an unlikely edge case.

As you might have guessed, we now have to edit Scene_Map. Add the following functions anywhere:

  def get_event_width(event)
    return @spriteset.get_event_sprite(event).width
  end
  def get_event_height(event)
    return @spriteset.get_event_sprite(event).height
  end

Often, you’ll find that the Scene_* classes contain the down-and-dirty details of the game engine, while the Game_* classes contain a higher-level “logical” set of data points. Why, I’d bet we could fix the screen jitter we discovered earlier by simply updating talkboxes here instead of in Game_Map. This is completely correct. Find the following line in Game_Map’s “update” method:

    update_talkboxes

…and remove it. Then, add the following line to Scene_Map’s “update” method, right after “@message_window.update”.

    $game_map.update_talkboxes   #Update NPC talkboxes

That’ll fix the 1-pixel jitter we noticed earlier. Why? My guess is that “@spriteset.update” (which occurs after “$game_map.update”) does a minor pixel correction on every sprite that’s scrolling. For Game_Map’s objects, this tiny offset isn’t important.

By the way, a lot of game objects have both a Game_* and a Sprite_* representation. We might consider doing the same thing for our windows; then we wouldn’t need this minor hack. However, I don’t think it’s important.

Add the following function to Spriteset_Map. The explicit loop this function contains, by the way, is the reason we cache @offset_y.

  def get_event_sprite(event)
    for character in @character_sprites
      return character if character.for_event?(event)
    end
  end

Finally, add the for_event? function to Sprite_Character:

  def for_event?(event)
    return event == @character
  end

All this is rather hackish, but it’s self-contained, so I don’t consider it such a sin. More importantly, it makes our talkboxes work! Create a few NPCs and give them long-ish textboxes. Have them walk around if you want. Start a new game and enjoy the much, much nicer talkboxes (note the walking cow).

Our New Boxes Auto-Size and Don't Jitter When You Walk.

Our New Boxes Auto-Size and Don't Jitter When You Walk.

Don’t worry about the fact that boxes appear on the map for a split second before moving above their respective NPCs; we’ll eventually start them all “closed”, which will fix this problem.

Step 2: Opening Boxes When In Range

We’ve come a long way, but  our talkboxes still need some major work. For one thing, it doesn’t make sense to show them all on-screen at once. Besides introducing an unnecessary bottleneck, this is also rather boring to look at from the player’s perspective. So, let’s start all talkboxes “closed” and then “open” them when an NPC steps into the farthest visible tiles. (We’ll close them if he steps out of bounds).

This will require oodles of code to be written. Phase one will be to make the windows open and type their text; the second phase will invoke this code when an NPC steps into bounds. We will borrow most of our new code from Window_Message. Here’s our new class:

#A version of a Message which appears over a single
# NPC’s head. Supports most text tags, and auto-sizing.
class Talkbox_Window < Window_Base
  #RPG Maker defaults
  MAX_WINDOW_WIDTH = 544
  MAX_WINDOW_HEIGHT = 416
  WINDOW_PADDING = 32

  #Create a new Talkbox_Window.
  #  @param npc – The GameEvent above which we show our talkbox
  #  @param txt – A String (or array of strings) which
  #               represents our message. (Arrays aren’t yet
  #               supported).
  def initialize(npc, txt)
    #Figure out a proper size, and initialize it offscreen
    #  Height is +3 to allow for g/p/y/etc.
    my_size = $Debug.contents.text_size(txt)
    super(0, 0,
       my_size.width + WINDOW_PADDING,
       my_size.height + 3 + WINDOW_PADDING)

    #Put off setting this variable until later
    @offset_y = -1

    #On top and invisible
    self.z = 90
    self.visible = false

    #Fully shrunken, and neither opening nor closing.
    self.openness = 0
    @opening = false
    @closing = false

    #Pause after opening? Note that we can’t use
    # self.pause; that'll set a bouncing graphic
    @passive_pause = false

    #Reset variables used to track the state of the
    # drawing routine
    @text = nil       #Our text buffer
    @contents_x = 0   #Next character’s X
    @contents_y = 0   #Next character’s Y
    @line_count = 0   #Lines drawn so far
    @wait_count = 0   #How long to pause (for \., etc.)
    @line_show_fast = false

    #Save our text-to-draw
    @orig_texts = Array(txt)
   end

  #
  # New functionality for text processing
  # 

  #Frame Update – All opening/closing/typing happens here
  def update
    #Handle opening/closing
    super

    #All other actions must wait for a fully-opened window
    unless @opening or @closing
      if @wait_count > 0
        #We’re pausing _within_ the text stream
        @wait_count -= 1
      elsif @passive_pause
        #The window is fully open; leave it open
        # (optionally, we can add code to scan for Input)
      elsif @text != nil
        #There's text we haven’t shown yet
        update_message
      elsif continue?
        #Open the window and show its text
        start_message
        open
      end
    end
  end

  #Start a new line of text
  def new_line
    @contents_x = 0
    @contents_y += WLH
    @line_count += 1
    @line_show_fast = false
  end

  #Should we be displaying the next message?
  # For now, the answer is “always”, but you can add your
  # own control code if you like
  def continue?
    return true
  end

  #Align all our text for processing into a single string with
  # embedded control characters
  def start_message
    @text = ""
    for i in 0...@orig_texts.size
      @text += @orig_texts[i].clone + "\x00"
    end
    convert_special_characters
    new_page
  end

  #Replace all special letters with single-character
  # equivalents. (Copied verbatim from Message)
  def convert_special_characters
    @text.gsub!(/\\V\[([0-9]+)\]/i) { $game_variables[$1.to_i] }
    @text.gsub!(/\\V\[([0-9]+)\]/i) { $game_variables[$1.to_i] }
    @text.gsub!(/\\N\[([0-9]+)\]/i) { $game_actors[$1.to_i].name }
    @text.gsub!(/\\C\[([0-9]+)\]/i) { "\x01[#{$1}]" }
    @text.gsub!(/\\G/)              { "\x02" }
    @text.gsub!(/\\\./)             { "\x03" }
    @text.gsub!(/\\\|/)             { "\x04" }
    @text.gsub!(/\\!/)              { "\x05" }
    @text.gsub!(/\\>/)              { "\x06" }
    @text.gsub!(/\\</)              { "\x07" }
    @text.gsub!(/\\\^/)             { "\x08" }
    @text.gsub!(/\\\\/)             { "\\" }
  end

  #Start a new “page” of input on the current message
  # box’s surface. Right now, we only have one page of
  # input, but there’s no reason you can’t add more.
  def new_page
    #Clear the background bitmap
    self.contents.clear

    #Reset our drawing state variables
    @contents_x = 0
    @contents_y = 0
    @line_count = 0
    @line_show_fast = false
    @passive_pause = false

    #Reset our text color to the default
    contents.font.color = text_color(0)
  end  

  #Called when a message has been printed in its entirety
  # (used below)
  def finish_message
    @passive_pause = true
    @wait_count = 10
    @text = nil
  end

  #This is the main text processing loop of the talkbox window.
  # It works by slicing one character at a time off of @text,
  # and either reacting to it as a control character,
  # or displaying it.
  def update_message
    loop do
      #Get next text character
      c = @text.slice!(/./m)
      case c
        when nil
          #There is no text waiting to be drawn
          finish_message
          break
        when "\x00"
          #Our custom "newline" character
          new_line
          #Multiple pages; left in for your reference
          #if @line_count >= MAX_LINE
          #  unless @text.empty?
          #    self.pause = true
          #    break
          #  end
          #end
      when "\x01"
        #\C[n]  (text character color change)
        @text.sub!(/\[([0-9]+)\]/, "")
        contents.font.color = text_color($1.to_i)
        next
      when "\x02"
        #\G  (gold display) -ignore
        break
      when "\x03"
        #\.  (wait 1/4 second)
        @wait_count = 15
        break
      when "\x04"
        #\|  (wait 1 second)
        @wait_count = 60
        break
      when "\x05"
        #\!  (Wait for input)
        break
      when "\x06"
        #\>  (Fast display ON)
        @line_show_fast = true
      when "\x07"
        #\<  (Fast display OFF)
        @line_show_fast = false
      when "\x08"
        #\^  (No wait for input)
        break
      else
        #Normal text character
        contents.draw_text(@contents_x, @contents_y, 40, WLH, c)
        c_width = contents.text_size(c).width
        @contents_x += c_width
      end
      break unless @line_show_fast
    end
  end

  #
  # The rest of our functionality is unchanged
  #

  #Call this to update the talkbox’s position over an NPC
  def update_pos(npc)
    #Store the npc’s height if we haven’t done so yet
    @offset_y = npc.sprite_height if @offset_y==-1

    #Update this talkbox’s position
    self.x = npc.screen_x - self.width/2
    self.y = npc.screen_y - self.height - @offset_y
  end

  #Get the width/height of the client area
  def draw_width
    return self.width - WINDOW_PADDING
  end
  def draw_height
    return self.height - WINDOW_PADDING
  end
end

What a whopper! Fortunately, this is fairly easy to understand: update() determines whether or not we should be calling update_message(), which reads letters one-by-one. All other methods are helpers, and their functions are relatively simple. I’ve got to hand it to the RPG Maker VX team: their code might be a bit cluttered, but it’s very easy to follow and understand. I’m consistently amazed at how sensible it is to hack the RPGMVX engine.

Now, add the following code in Game_Event’s update() method, right below check_event_trigger_auto:

#Update NPC talkbox
@npc_talkbox.update if @npc_talkbox != nil

This is much easier to understand; we just needed to hook up our frame update method when the Game Event calls its update method. Finally, add a new event to your map, with the following “Script” command:

for event in $game_map.events.values
  tkb = event.npc_talkbox
  next unless tkb
  tkb.openness = 0
  tkb.start_message()
  tkb.open()
end

Since windows stay open forever, we need a way to “re-type” the window, to make sure our system is working properly. Since we didn’t bother writing such a method (we will later), we just hooked it up manually for now.

Run your program; talk to your new NPC any time you want to reset every window. Ah…. Isn’t that nice? As an added benefit, our windows no longer show up on the title screen, since their “openness” only gets updated in the main game loop.

These TalkBoxes Show Their Messages Just Like a Normal RPGMVX Show_Message Box.

These TalkBoxes Show Their Messages Just Like a Normal RPGMVX Show_Message Box.

Before we make these talkboxes appear on demand, we have to decide when exactly we want to show them. I chose to wait for a talkbox to be fully onscreen vertically, and halfway onscreen horizontally before starting to open it. I made this decision partly because boxes are centered over NPCs, so a talkbox will now appear slightly before or after its respective NPC becomes visible. Add the following function to your Window_Talkbox class; it checks whether or not this box should be opened:

  #Is this box within the vertical boundaries, and at
  # least halfway within the horizontal boundaries?
  def in_range?
    #Store some marker variables
    mid_x = self.x+self.width/2
    min_y = self.y
    max_y = self.y+self.height

    #Check bounds
    return true if min_y>=0 and mid_x>=0 and
                   max_y<=MAX_WINDOW_HEIGHT and
                   mid_x<=MAX_WINDOW_WIDTH
    return false
  end

As you should have noticed, this function relies on the update_pos() function being called first. And, if you think about it, update_pos doesn’t work unless called in the Scene_* routine; calling it in the context of Game_* is a mistake. So, instead of juggling a whole slew of function calls, we’ll just append our code to update_pos(). Your update_pos function should now look like this:

  #Call this to update the talkbox’s position over an NPC
  def update_pos(npc)
    #Store the npc’s height if we haven’t done so yet
    @offset_y = npc.sprite_height if @offset_y==-1

    #Update this talkbox’s position
    self.x = npc.screen_x - self.width/2
    self.y = npc.screen_y - self.height - @offset_y

    #Now, show/hide we show this box?
    if @onscreen != in_range?
      unless @onscreen
        retype_text()
      else
        close_window()
      end
    end
  end

This relies on a new variable, @onscreen. Add it to your init function (anywhere)

@onscreen = false  #In-range?

Finally, you’ll need the functions retype_text() and close_window(). Add them anywhere:

def retype_text
  @onscreen = true
  openness = 0
  start_message()
  open()
end

def close_window
  @onscreen = false
  close
end

This should do the trick. The only thing left to do is to find the following lines in update’s “elsif continue?” branch and comment them out:

  # start_message
  # open

…otherwise, the window will always open itself, regardless of its position onscreen. Run your code:

Our Boxes Now Open When In Range, Completing Our Original Goal.

Our Boxes Now Open When In Range, Completing Our Original Goal.

The nice thing is, since we check this every tick (instead of just when the NPC moves) it should work properly for teleporting to a new map and teleporting NPCs around the map.

Note: The documentation states that if a Window’s “openness” value is less than 255, its “contents” Bitmap won’t be displayed. I am kind of assuming that, if its “openness” is 0, the windowskin also won’t be displayed, and processing will be minimal. I feel this is a safe assumption, but I want to make it clear that, if you experience a performance hit, you might want to look here first.

Possible Improvements

This tutorial was rather longish, so I’ll only give you two assignments this week.

  • It’s well known that inheritance simplifies code reuse. Yet, rather than reusing the code in Window_Message, we just copied it and deleted what we didn’t want. Wouldn’t it be better to abstract it into a common superclass, and have both Window_Message and Window_Talkbox subclass this new class? For our tutorial: no. Extracting this code could have broken both normal messages and our talkboxes. Our lightweight class only risks breaking itself, which makes it ideal for development. Now that we’re done proving the concept, however, it would be a good idea to create a common superclass. That’s your first assignment.
  • We copied the convert_special_characters() function directly from Window_Message, but closer inspection makes it clear that only a small number of backslashed “special letters” are used. Might we extend the list of possible escape sequences? Since we extract the text for our message balloon directly from a ShowMessage event, this provides a very elegant way of specifying additional properties of talkboxes. For example, we might use \{ACCEPT} to mean “this talkbox responds to the Accept key”, and \{EVENT:2} to mean “call event 2 when the user presses Accept”. Perhaps \{CLOSE:10} could means “close this talkbox 10 ticks after all text has been displayed” —indeed, we’ll need a lot more control if we’re to use these things for cutscenes.

June 1, 2009

[REV-1] Current Exciting Hacks

Blog Info: I’m in a bit of a rut, time-wise. My latest project just developed a huge bug, and my current research project is also eating up a lot of time. I could tell you my plans for future blog posts, but I’m notorious for changing my mind. Instead, I’ve got a much better idea!

Back-story: We’ve been hacking away for a while; it’s time to take a second and reflect on hacks that other people in the gaming community are working on. To me, this is always a rewarding experience: you’ll find a lot of really creative and exciting ideas out there, and you’ll be able to refine your own ideas a bit too. This week, we’ll focus on ROM hacks, particularly my penchants of Mario and Zelda. Where possible, I’ll provide videos.

Super Mario Odyssey
Summary: A thorough hack of Super Mario World that somehow presents itself as “just a simple hack”.

My Thoughts: Except for a certain two player modification, this has got to be my favorite ROM hack. The reason’s simple: it’s the only one that’s well designed! Level design on most hacks is idiotic; the goal seems to be to get the player lost, confused, and frustrated. Let’s not forget that one of the most fun parts of the original SMW was improving to the point where you could run through the early levels with ease. Super Mario World Odyssey is a nice mix of challenging, fun, and epic grandness. It’s a shame that the developer hasn’t spoken up in years; this is one of the few hacks I’m actually looking forward to.

The Legend Of Zelda: Lost Isle
Summary: Technically not a hack. Based off a Zelda engine that was first reverse-engineered and then enhanced. Very thorough, very pretty.

My Thoughts: Apologies if I appear nonplussed, but I find even the copyright holders mess up Zelda quite often -so how can hobbyists hope to get it right? The dungeons in this game are needlessly confusing, and the overworld is basically dead. That said, compared to Parallel Worlds, this game is not bad. Sure, it’s got a huge world map, but it also implemented a useful teleport system to ease the burden of long treks through the jungle. And the world really retains the “feel” of the Zelda universe, which is clearly a credit to the designer’s skill. No offense intended to the Parallel Worlds team, by the way; that hack is vast and professional, but its general design is uninspiring and amateurish. Consider it constructive criticism: the game would be an excellent first draft, in the sense that seriously re-tooling half of it would do wonders.

SMW Zelda (Un-Named Hack)
Summary: For completeness, this hack combines the previous two games! It’s a 2-D, Mario-ish Zelda game!

My Thoughts: I’m cautiously optimistic about this one. I’ve always wondered what a 2-D Zelda game would be like, but I always thought it’d be more like Hannah and the Pirate Caves. Still, I like the hacker’s take on this one. The Deku Leaf is not just a weakly disguised cape; Deku Nuts (not shown) go straight forward but bounce at an angle, and the shield repels enemies. I found Legend of Princess to be a disappointing attempt at a 2-D Zelda (except fighting Shadow Princess; that was amazing), but so far, “SMW Zelda” seems like it’s blending both games to come up with something new. I really, really hope this one turns out good.

Super Mario World: The Second Reality Project 1
Summary: An “old-school” Mario World hack, which means the Overworld and the blockset haven’t been modified; only the levels have been changed.

My Thoughts: I dislike most Mario hacks. Super Demo World was nice to look at and modestly creative, but it gave me headaches. Various other hacks, requiring you to pull off such stunts as spin-jumping while holding a P-switch, were mere tributes to their designers’ egos. However, SRP1 offers something none of those do: an “easy mode” which is actually playable without savestates. It’s incredibly challenging, but so far (World 6) I wouldn’t say it’s unfair. (Editor’s note: Worlds 7+ are unfair.) And because it doesn’t use nonstandard hacks, I can take it on the go. If all developers took the time to step back and actually try their game out as a player instead of a developer or hobbyist, I think we’d have a lot more easy mode hacks, and homebrew would be all the better for it.

Various Spriter Sonikku Tricks (not a hack)
Summary: Some fascinating (but simple) ideas from a hacker.

My Thoughts: As a bit of a parting shot, have a look at this guy’s hack videos. It seems like he strives for simple, elegant hacks with lots of stopping power. Consider the video above, which allows bombs to destroy an on/off switch. I can immediately think of two clever tricks (and one gimmick) leveraging this ability. Although I don’t particularly like his games, Spriter Sonikku’s approach to development is geared towards maximum effect. Excellent~

Further Work
The Second Reality Project: Reloaded is a re-write of TSRP1 which changes very little in the way of level design and adds a much more coherent backstory.

I have a theory that good designers, when constrained, can sometimes produce better work than when they have full freedom. The Reloaded hack, for example, encouraged FPI productions to reuse older levels and eschew most custom blocks and scripts. In the end, I found this game better than both TSRP1 and TSRP2. So, in the spirit of being creative under extreme pressure, here’s your assignment for this week:

  • Take a hack you enjoy somewhat, and pretend that your friend has asked you to spruce this up for a 1.1 release. Design (on paper; no code necessary) an improved hack which modifies very little from the original game but feels much more consistent overall. Try to write a few sentences describing the overall feel of the game, and then pinpoint parts of it that clash with this. Make sure you only suggest improvements which you know are possible. Try to limit yourself; pretend that you have a few friends to help you (for time-consuming tasks like replacing graphics), but you only have 6 months to develop the new version.

Design documents are due on my desk by Monday, although to quote a random internet source, I will also accept the excuse “I’m too lazy to do it”.

April 11, 2009

[EL-1] Game Event Notifier

Blog Info: I’ve made a habit of switching topics often on this blog. This might frustrate some of my readers, but believe me, it’s much healthier than constantly revising one project over and over. Besides, since none of these projects are real money-makers, we might as well build as broad a tool-set as possible. So in this guide, we’ll learn a bit about Win32, one of the oldest stable windowing toolkits in existence.

Back-story: Everyone has their MMO of choice. I play Eternal Lands because it’s very, very slow-paced, so I have no chance of getting hooked on it. It’s also quite balanced, and has beautiful scenery and a believable set of systems. You could really tell a story using that world. That said, some minor in-game features really bother me. For example, when you are gathering (“harvesting”) vegetables and minerals, you often unearth some nasty surprise that causes your character to pause. Since harvesting is meant to be the kind of thing you minimize the game for, this means that you’ll be constantly bringing the game back just to check if you’ve been interrupted.

Goal: Figure out a way to notify players when they stop harvesting. Perhaps a pop-up notification like Google Talk provides?

A note about Linux: Despite the fact that Eternal Lands is actually easier to compile under Linux, I’ll be developing this plugin under Windows. GTK fans should have enough knowledge to be able to port this example to Linux. My reason for doing this is simple: on my machine, Eternal Lands runs better on Windows. I’m all for equality, but I’m not going to handicap myself for the sake of fairness.

Getting the Source
Eternal Lands is rather unique in that the client is open-source. I find this rather trusting of the developers, although presumably they did it so that they could get features patched in by the user community. This makes our project a lot easier, so let’s grab the source right away. You’ll need Tortoise CVS. (Note that Tortoise CVS works fine on Windows Vista, as long as you don’t try to perform a CVS Checkout in a protected directory like C:\windows.) The source is located at the project’s BerliOS repository. To access it, create a folder called “EL Client” somewhere on your system. Open the folder, right-click, and choose “CVS Checkout…”. Copy and paste the CVS Root from the source page into the corresponding checkout field; it’ll look something like this:
:pserver:anonymous@cvs.elc.berlios.de:/cvsroot/elc
Then, enter elc for the module name. You can now click Ok:

Entering the CVS Root and the Module Name enables the "Ok" button.

Entering the CVS Root and the Module Name enables the "Ok" button.

The entire project contains lots of files, so you’ll have to wait a while for them all to download. While you’re at it, make sure you’ve downloaded and installed the Windows client of Eternal Lands from the main site. (You might consider the Down Them All plugin for Firefox, as the Eternal Lands site will occasionally yahtzee your connection.) This will make running the .exe you compile much easier.

Theory, Approach, and Not Getting Banned
While our downloads chug along their merry way, let’s talk design. First of all, what’s our end goal? That’s easy: “When the client shows the message You have stopped harvesting we should show a popup near the system tray if the Eternal Lands client is minimized.”

Next question: is this legal? Modification of the client code is fine, since the license is very liberal about the source. Actually using it to connect to the game requires that we subscribe to the terms and conditions of Eternal Lands. The only one that applies to us is:

[No] macroing, automating, scripting, exploiting bugs...

However, I don’t think we’re in violation of this. We’re not macroing, and the only thing we are automating or scripting is the use of Alt+Tab. I don’t think Entropy would insist that we constantly check up on our harvesting progress, especially since shell scripts in Linux to check the message logs have been received no hostility on the forums. In fact, this notification actually maximizes the ratio of player-controlled time to time wasted standing still.

Well, it’s nice that we won’t get banned for doing this, but how will we actually program it? There are two approaches I can see: either modify the game to display the window itself, or create a separate program that intercepts packets sent to Eternal Lands and determines if any of these packets represent the You have stopped harvesting message. There are pros and cons to each approach:

Modifying the Game Client
Pros

  • You won’t lose any speed. Checking each message and responding only for a few specific ones will not affect performance.
  • Modifying the code itself is easy; you won’t have to play with firewalls or packet interceptors.

Cons

  • You’ll have to be able to compile the client. For some games (especially cross-platform ones like Eternal Lands) this can be a show-stopper; it’s difficult!
  • You’ll need to re-apply your fix every time a new game client is released. This should be easy, but major releases (especially ones that use new libraries) might bring up the issue of compiling again.

Intercepting Packets
Pros

  • No modification to the client is necessary. You can easily start the client with or without this feature enabled.
  • When a new client is released, your code might continue to work! In fact, it will only fail if the connection protocol or information protocol changes, which is something the developers will probably avoid.
  • You can develop in your favorite IDE; you don’t have to compile their code first. Consider: let’s say that we’re running a Windows game in Wine; compiling the client could be a nightmare!
  • The result is slightly easier to port than modifying the code.

Cons

  • The concept is more difficult; we’re dealing with packet nonsense instead of strings.
  • If your code isn’t fast, it might cause your game to lag, since you’re dealing with all packets, not just the rare console message.

Since this is our first tutorial on Eternal Lands, we’ll modify the client code directly. In our next lesson, we’ll create a packet interception program, and polish up the interface so it looks real shiny-like.

Compiling the Code
The Official Eternal Lands Client Development Site has instructions for compiling the client using Dev C++. I followed these instructions and they worked perfectly. If this link is ever broken, you’ll have to figure it out on your own, or you can try compiling the client on Linux and re-working this tutorial from there.

You might want to remove debugging symbols from the compiled executable; this makes it about 77% smaller. To do that, add the line strip –s $(EXE) to the end of your $(EXE) target. In other words, in Makefile.win, find the following:
$(EXE): main.o $(TMP_LIB)
$(LINK) $(CFLAGS) $< -L. -lelc $(LDFLAGS) -o $(EXE)

..and replace it with:
$(EXE): main.o $(TMP_LIB)
$(LINK) $(CFLAGS) $< -L. -lelc $(LDFLAGS) -o $(EXE)
strip -s $(EXE)

Although the original 11.1MB isn’t too big, the resultant 2.56 MB is small enough to email to a friend.

To run the program, you should follow their instructions for running a local copy. In particular:

  • Copy the contents of the “Eternal Lands” directory from the official client release (e.g., C:\Program Files\Eternal Ladns) to a new directory called “Release” in the “EL Client” directory where you saved the source.
  • After you correctly compile the source in Dev C++, you’ll have a file el.exe in the “EL Client” directory. Copy that into the “Release” directory, over-writing the file that’s there.
  • Copy the files libcal3d-12.dll and alut.dll from C:\Dev-Cpp\dll and C:\Dev-Cpp\lib (respectively) to Release, over-writing the files there.
  • Run el.exe in the “Release” directory.
  • Wow! I don’t know about you, but for me, simply re-compiling the client caused it to run about 30% faster! This is particularly astounding, since I wouldn’t expect a naïve build to optimize better than the developer’s build. Someone should try compiling this thing in Visual Studio with PGO and seeing if we can make it even faster.

    Our Client Compiles, And It Looks Great!

    Our Client Compiles, And It Looks Great!

    Adding a Window at Startup
    The next logical step is to create a simple window as a proof-of-concept, and show it when Eternal Lands starts up. We’ll be using pure Win32 C to code this.
    In Dev C++, click “File->New->Source File” and choose “Yes” when asked if you want to add this file to the current project. Then, immediately “save” the file as taskbaralerts.h, with a type of “Header Files”. Do the same for another file, taskbaralerts.c, of type “C source file”. These will contain the bulk of our implementation; it’s a very good idea to keep as much code as possible out of the official EL source files.

    Enter the following into taskbaralerts.h:
    #ifndef TASKBARALERTS_H
    #define TASKBARALERTS_H

    //Includes
    #include <stdio.h>
    #include <windows.h>

    //Function prototypes
    LRESULT CALLBACK AlertsWndProc(HWND hwnd, UINT msg, WPARAM wParam, LPARAM lParam);
    void initAlertsWindow(HINSTANCE hInst, char *windowClassName);

    #endif //TASKBARALERTS_H

    The #ifndef/#define/#endif triplet is known as an Include Guard, and it prevents us from defining multiple versions of “initAlertsWindow”. Now, any file that wants to use these functions can simply include taskbaralerts.h and everything will run smoothly.

    Of course, we need to actually implement these somewhere. Add this to taskbaralerts.c:

    #include "taskbaralerts.h"

    //Global variable
    HWND alertsWindow = NULL;
    HWND hStatic = NULL;
    BOOL windowVisible = FALSE;
    HINSTANCE myHInst = NULL;
    HBRUSH bkgrdBrush = NULL;

    //Message-handling callback
    LRESULT CALLBACK AlertsWndProc(HWND hwnd, UINT msg, WPARAM wParam, LPARAM lParam)
    {
    switch(msg)
    {
    case WM_CREATE:
    {
    hStatic = CreateWindowEx(WS_EX_LEFT, “Static”, “”,
    WS_CHILD | WS_VISIBLE,
    5, 10, 180, 50,
    hwnd, NULL, myHInst, NULL);
    SetWindowText(hStatic, “EL Notification Area”);
    break;
    }
    case WM_LBUTTONDOWN:
    {
    //Hide our window
    windowVisible = FALSE;
    ShowWindow(hwnd, SW_HIDE);
    break;
    }
    case WM_CLOSE:
    {
    //Over-ride: Hide our window
    windowVisible = FALSE;
    ShowWindow(hwnd, SW_HIDE);
    break;
    }
    case WM_DESTROY:
    {
    //Close the window for good.
    PostQuitMessage(0);
    break;
    }
    default:
    return DefWindowProc(hwnd, msg, wParam, lParam);
    }
    return 0;
    }

    void initAlertsWindow(HINSTANCE hInst, char *windowClassName)
    {
    //Declare c-style variables
    RECT r;
    WNDCLASSEX wc;

    //Save
    myHInst = hInst;

    //Necessary?
    if (alertsWindow!=NULL) {
    return;
    }

    //Make a brush
    bkgrdBrush = CreateSolidBrush(RGB(0, 0, 0));

    //Set a window class’s parameters
    wc.cbSize = sizeof(WNDCLASSEX);
    wc.style = 0;
    wc.lpfnWndProc = AlertsWndProc;
    wc.cbClsExtra = 0;
    wc.cbWndExtra = 0;
    wc.hInstance = hInst;
    wc.hIcon = LoadIcon(NULL, IDI_APPLICATION);
    wc.hCursor = LoadCursor(NULL, IDC_ARROW);
    wc.hbrBackground = bkgrdBrush;
    wc.lpszMenuName = NULL;
    wc.lpszClassName = windowClassName;
    wc.hIconSm = LoadIcon(NULL, IDI_APPLICATION);
    if(!RegisterClassEx(&wc)) {
    alertsWindow = NULL;
    return;
    }

    //Get the desktop
    GetWindowRect(GetDesktopWindow(), &r);

    //Create a handle to the window
    alertsWindow = CreateWindowEx(
    WS_EX_TOPMOST | WS_EX_NOACTIVATE,
    windowClassName,
    “Eternal Lands Notification”,
    WS_CAPTION | WS_SYSMENU, //Simple
    r.right-200, r.bottom-100, 200, 100,
    NULL, NULL, hInst, NULL
    );
    if (alertsWindow==NULL) {
    return;
    }

    //TEMP: Show the window
    windowVisible = TRUE;
    ShowWindow(alertsWindow, SW_SHOW);
    UpdateWindow(alertsWindow);
    //TEMP end
    }

    That’s a lot to digest. Let’s start at initAlertsWindow(), which the main game should call when it first loads. First, we check if alertsWindow is not NULL, which could happen if, say, the game restarts itself by re-running main(). Next, we fill out the parameters of our WNDCLASSEX struct. (The MSDN has very good documentation (except, occasionally, when it’s totally wrong!) on its Win32 API, so I’ll be linking to that quite a bit.) The parameter of interest is lpfnWndProc, which points to the function which handles messages for this window. We set it to AlertsWndProc. This means that every time a message is dispatched to our alerts window, the AlertsWndProc function handles it.

    Now, we try to register this window class. The function call might fail if, say, Eternal Lands has already created a window class with this name. At any stage along the way, if a function call fails, we simply return. Now we are ready to create the window. However, since we want to position it on the lower-right corner of the screen, we’ll need to know how big the desktop is. So, we call GetWindowRect(GetDesktopWindow(), &r), which stores the desktop’s dimensions in r. Now, we can call CreateWindowEx. Parameters of interest include:

    • The dwExStyle is used to describe what type of window we want. WS_EX_TOPMOST means that we want our window to appear above all other non-topmost windows, and WS_EX_NOACTIVATE means that we want our new window to show, but not to take keyboard focus or have a section on the taskbar. This makes sense; if we are typing in Microsoft Word when we stop harvesting, we want the notification to appear above Word, but we don’t want it to take the focus away from word.
    • The dwStyle parameter describes what our window looks like. We give it a WS_CAPTION, and add a WS_SYSMENU so that the user can “close” the window if he wants to. Nothing fancy here.
    • The four paraemeters x, y, nWidth, and nHeight position our window on the lower-right corner of the screen. You can fiddle with it if you want to display it above the taskbar; it doesn’t really bother me.

    That’s all there is to creating the window, and showing it is easy enough. The AlertsWndProc function simply switches through the variety of message types we can receive, returning “0” if it sees nothing of interest (which tells Windows to handle the message itself). In the WM_CREATE message, we add a static sub-window (called a “Label” in other languages) so that we can potentially display multiple messages in the future. The WM_LBUTTONDOWN message tells our window to hide itself; this lets the user simply click on the notification window to make it go away. We also intercept the WM_CLOSE message, because we don’t want the “X” button to actually close the window. WM_DESTROY functions normally for destroying a window, since closing Eternal Lands will also close all of its windows. (Don’t worry, this code won’t spawn any hidden processes that you’ll have to Ctrl+Alt+Delete).

    There’s a few things left to do before we actually run this:

    1. In Makefile.win, find the line that begins with COBJS= and add taskbaralerts.o in between tabs.o and text.o. If you were paying attention you’d notice that although we include taskbaralerts.h in several places, nothing links to taskbaralerts.c! This addition ensures that the linker can find your functions.
    2. In main.c, add #include “taskbaralerts.h” at the top of the file. If you want, you can add an #ifdef check so that the client will still compile under Linux, but it’s not really necessary. Now, find the line init_stuff(); and add initAlertsWindow(hInst, “EL_Taskalert”); after it. This will initialize our taskbar alert code right before Eternal Lands starts rendering the main game.
    3. Unfortunately, hInst isn’t defined. Find the line char *win_command_line; at the top of the file, and add HINSTANCE hInst; after it. Then, find the function APIENTRY WinMain and add hInst = hInstance; after int argc; —that was easy, right?

    So, shall we run this code? Compile it, and copy el.exe into the Release directory. Run it, and you’ll see our window pop up about halfway through the boot sequence. If you’re in full-screen mode, this will be very obvious.

    Our Notification Is Created When EL Starts

    Our Notification Is Created When EL Starts

    Intercepting Messages
    Delete the lines we wrapped with the “TEMP…TEMP end” comments so that our window won’t show. It’s time to start fiddling with EL’s messages. In Dev C++ click “Search->Find In Files”. This allows us to search every file in our project for some text, and is indispensable when programming in C. Let’s try to find a message —any message— and go from there. What’s the most unique word in Eternal Lands? How about “radon”? Type it in and click “Ok”. The highlighted result looks particularly promising; double-click and you’ll be brought to the code that contains this —yep, it’s a big array of message strings:

    We've Found The Radon Message (It's Misspelled As "Radeon" In One Location)

    We've Found The Radon Message (It's Misspelled As "Radeon" In One Location)

    The name of the array is temp_event_string; scan the file a bit and you’ll find that this gets sprintf’d into search_str, replacing the %s with username_str. Well… that’s a tangential bit of useful information. Do a “Find In Files…” on username_str and you’ll see that it’s defined in interface.h. So, if we include interface.h, we can access our player’s name. Getting back to the problem at hand, do a “Find in Files…” on catch_counters_text (the name of the function which accesses search_str), and you’ll see that text.c calls this in filter_or_ignore_text(). We just completed a bottom-up search; if you did a top-down search starting from the TCP layer, you’d end up at the same place. Go to the top of the function, under #endif // NEW_SOUND, and put in a call to check_taskbar_alerts(text_to_add); —making sure to #include “taskbaralerts.h” somewhere at the top of the file. Now, switch back to taskbaralerts.h and add a function prototype:
    void check_taskbar_alerts(char * text);
    In taskbaralerts.c, add #include “interface.h” at the top of the file so we can access our player’s name. Now, before we go any further, we have a small problem. You see, the mess you receive isn’t exactly equal to “You stopped harvesting.” There’s a character at the beginning which specifies the color of the string. We could simply strcmp on “\x7FYou stopped harvesting.”, but this breaks our code if the color of this message changes. What we want is a function named “stringcontains”, which we’ll write next. (Of course, if a player now PMs you with the message You stopped harvesting, this will also set off your alarm. However, I think it’s better to react to this case then try to prevent it.) Add the following code at the top of taskbaralerts.c:

    BOOL strcontains(char* src, char* pattern)
    {
    //Init some tracking variables
    int i;
    int len;
    int patIndex = 0;
    int patLen;

    //Loop
    len = strlen(src);
    patLen = strlen(pattern);
    for (i=0; i<len; i++) {
    //Have we reached a non-match?
    if (src[i]!=pattern[patIndex]) {
    patIndex = 0;
    }

    //Continue tracking our pattern
    if (src[i]==pattern[patIndex]) {
    if (patIndex==patLen-1) {
    return TRUE;
    } else {
    patIndex++;
    }
    }
    }
    return FALSE;
    }

    This function is simple enough; whenever it matches the first letter of the pattern with the current index in the string, it simply assumes it has a match, and resets to zero if that assumption is broken. This will fail if you’re looking for ABAC in the string ABABAC, so you can re-write it later (when we test for usernames) if your username is something funky like that.

    Now, let’s hook this function up:

    //Bring up our window?
    void check_taskbar_alerts(char * text)
    {
    //Can we track this at all?
    if (alertsWindow==NULL) {
    return;
    } else if (windowVisible==TRUE) {
    windowVisible = FALSE;
    ShowWindow(alertsWindow, SW_HIDE);
    }

    //Have we stopped harvesting?
    if (strcontains(text, “You stopped harvesting”)==TRUE) {
    //Prompt the user
    SetWindowText(hStatic, “You stopped harvesting.”);
    windowVisible = TRUE;
    ShowWindow(alertsWindow, SW_SHOW);
    UpdateWindow(alertsWindow);
    }
    }

    Again, simplicity rules the day. This function checks each message as it is received. If the alertsWindow is still visible, that means the user forgot to close it, and we do that for them. Since users are prompted when they begin harvesting, this ensures that the window won’t give any false positives. We then use our new strcontains() method to check if our message looks something like “You stopped harvesting”. If so, we change the text of the window and make it visible. Compile the project and run it. It works! —but there’s one obvious flaw.

    Only Displaying the Notification if We’re Not Viewing Eternal Lands
    That’s right —the notification appears whenever we stop harvesting, even if we’re currently watching the Eternal Lands window. This is annoying as it grabs us out of full-screen mode. So what should we do? Well, the Eternal Lands window’s title bar contains the caption (Userame on main) Eternal Lands. The “main” changes if you switch servers, so let’s just use our strcontains function to search for the user’s name and the phrase Eternal Lands, and not show the window if both are present. Change our check_taskbar_alerts function to read:

    //Bring up our window?
    void check_taskbar_alerts(char * text)
    {
    //Can we track this at all?
    if (alertsWindow==NULL) {
    return;
    } else if (windowVisible==TRUE) {
    windowVisible = FALSE;
    ShowWindow(alertsWindow, SW_HIDE);
    }

    //Have we stopped harvesting?
    if (strcontains(text, “You stopped harvesting”)==TRUE) {
    HWND foreWnd = GetForegroundWindow();
    char wndTitle[200];
    GetWindowText(foreWnd, wndTitle, 200);

    //Does the window title contain the user’s name and the words “Eternal Lands”?
    if ((strcontains(wndTitle, “Eternal Lands”)==TRUE) && (strcontains(wndTitle, username_str)==TRUE)) {
    //We’re in the game! Do nothing
    } else {
    //Prompt the user
    SetWindowText(hStatic, “You stopped harvesting.”);
    windowVisible = TRUE;
    ShowWindow(alertsWindow, SW_SHOW);
    UpdateWindow(alertsWindow);
    }
    }
    }

    Try it out, and you’ll see that everything works just fine.

    Possible Improvements
    There’s a lot of potential for improvement, since we did such a hasty job designing our plugin. Here’s some particularly sore points; add your own and fix it up!

    • If we stop harvesting, the message displays properly. However, it will disappear if another message occurs before we maximize the game client. Moreover, if we see the message and then open the client and tell our character to walk towards a storage (a common response to a full inventory) the message will remain until another message preempts it. Both of these glitches imply that we were wrong to assume that we should hide the alert window when a non-harvesting message appears; rather, we should hide it when the client gains focus again.
    • It would be nice to have an icon in the status bar that our window is attached to.
    • We should probably give our EL notification box a more EL-ish feel. We could do this with some simple GDI programming.
    • We should be able to track more notifications, and display nice pictures related to their content. Like… if we find gold, etc. And, we could have our system tray icon flash bright red if we die (or green if we find a rare stone!).
    • In fact, all of our suggestions are edging towards a general re-design of the plugin we made. But in that case, I’d rather not program it in pure Win32 C. If you’re of a different mind, by all means implement these changes. Me, I’ll be brushing up on .NET, Java, and Python. See you next week!

    March 19, 2009

    [LZ-2] Bow & Arrow Quest

    Blog Info: These blog posts are supposed to take you about an hour to do, and me about three hours to write. That proved to be a gross underestimation –the prototype ROM for this post alone took at least three hours, and writing up step-by-step instructions has taken a few hours too. But I think we really accomplish something in this post.

    Back-story: We decided in the last post to modify Zelda3 to make the bow-and-arrow the primary weapon for the first part of the game. But how do we actually go about doing this? And, how can we make the first part of the castle more challenging for advanced players? An arrow-related puzzle is in order! But this means we’ll have to start learning how to use (shudder) the Dungeon Editor. (There is no Dragon Editor).

    Goal: Become proficient in using the Dungeon Editor, remove the sword, add the bow, and modify the first dungeon room you fall into to require the clever use of arrows to exit.

    Step 1: Understanding Backups

    Open a clean copy of Zelda3 in Hyrule Magic. Make SURE this is a copy, because I’m going to show you how simple it is to permanently muck it up. Expand “Dungeons” and click “Starting Location 03”, then click on the “More” button at the bottom of the dungeon screen (near “Starting Location”). See the radio button for “Upper Left”, next to “Horizontal Scroll”? Click that:

    Clicking This Radio Button Will Definitely Crash Zelda3.

    Clicking This Radio Button Will Definitely Crash Zelda3.

    …and then click “Ok” and save. Start the ROM…. and notice that it crashes after the name select screen. Ok, go back to the “More” button and change the setting back to “Upper Right”. Save, start the ROM…. it’s still failing to work.

    With one simple click, we have irreversibly corrupted our ROM. Sure, you can try to HEX it back to the status quo, but that could take hours. A better solution is to keep regular backups, and to do your most dangerous fiddling directly after you backup. Really, I’m not kidding when I say that you need to back up your working ROM far more often than usual.

    Step 2: Understanding the Dungeon Editor

    Open a clean copy of the Zelda3 ROM in Hyrule Magic. It’s time to play around with the dungeon editor. Open an MP3 player, too, for your sanity.

    Expand the “Dungeons” tab and browse down to “Starting Location 04”. Double-click, and you should see a familiar room: the entrance to the sewers. In the top-left corner of the window it says “Room 81”. Next to this are some arrows to switch to the next room in each of the four directions. Finally, there is a “Jump” button which allows you to go immediately to a room by number. Click on the arrows a few times to see where it takes you, and then click on “Jump”, type in “81” and hit “Ok”. You’re now back at the throne room.

    The Jump to Room Box

    The Jump to Room Box

    Close the dungeon view and scroll down to “Entrance 32” & open it. This is the room you fall into after pulling up that bush outside the castle. Actually, that’s somewhat inaccurate. This entrance only takes effect when you walk out of and then back into this room. It is the same room as the one you fall into (and the one you start from after saving) but depending on how you enter the room determines a number of minor things like where the camera starts. It’s important to remember this when you edit certain properties of this entrance. Make sure you always access it by double-clicking “Entrance 32”, unless I tell you to click “Starting Location 03”.

    Look at the box in the lower-right corner of the Dungeon Window. There are several options:

    • “1” – Layer 1 is where we do most of our editing. Doors can go here, too. Treasure chests often are placed here.
    • “2” – Layer 2 is required for some editing tricks (like bridges) –I won’t be using it here. Doors can also be placed here.
    • “3” – Pretty much only doors can go here. We’ll use this layer to make an entrance leading outside –that door has to go on this layer.
    • “Sprite” – Things like enemies, crystal switches, and NPCs go here.
    • “Item” – Items that can be placed on the ground or in pots go here.
    • “Block” – This setting lets you organize your pushable blocks. We won’t be using this.
    • “Torch” – This setting is for torches. We won’t use this, except to delete one torch.

    Click on “1”, and then look for the corner jutting out on the left side of the watery place where you land. When you click on it, you will see the following information displayed in the lower-right:

    Obj: 10D
    X: 2D
    Y: 11

    Right-click on this object, and choose “Remove”. You’ve successfully deleted the corner.

    Right-Click then "Remove" to Delete an Object

    Right-Click then "Remove" to Delete an Object

    Now click on the piece directly south of the one you deleted. Click & drag it left until its X co-ordinate reads “2A”. Now move the mouse over the top edge of the same piece and click+drag to “extend” it until its “Size” reads “03”.

    Unfortunately, the piece we moved is now under another piece. So, left-click on it, and then press “Ctrl+V” to bring it to the front. (Ctrl+B brings it to the back).

    Right-click anywhere and choose “Insert an Object”. Scroll up to object 108, click it, and choose “Ok”.

    Add an Object of Type 108

    Add an Object of Type 108

    It will appear roughly where you right-clicked; drag it to 2A,0B. Makes a nice corner, doesn’t it?

    Congratulations, we’ve made our first real change to the room! Save.

    Move the Wall to the Front with Ctrl+V

    Move the Wall to the Front with Ctrl+V

    We can describe what you just did using a simple table:

    Comment Before After
    wall x:2D y:11
    wall x:2D y:15 x:2A y:0F size:3 front
    wall obj:108 x:2A y:0B front

    That means “Delete object at location 2D,11”, then “Take object at 2D,15, move it to 2A,0F, resize it to 3 units, and bring it to the front”, then “Add a new object of type 108 at 2A,0B and bring it to the front”. It’s important that you understand this syntax; we’ll be modifying a lot of objects, and it would be horribly cruel to describe each one with a paragraph.

    The “Comment” column is an informal addition to help you keep your place.

    A note about resizing is in order. Sometimes, when you place a piece down (particularly one that stretches vertically) the piece will stretch to fill up all available vertical space. Please note that you can still resize it, you just have to grab the handle at the bottom and drag it all the way to the top. This is annoying, but not insurmountable.

    Step3: Enabling The Bow

    Back in Entrance 32, click on the “Sprites” radio button, then click on “Priest/Uncle”. Then, right-click on him, and choose “Remove”. Our Uncle will now no longer be there to give us the sword.

    Close Entrance 32 and open Entrance 00. Click once on the chest with the lantern. You’ll see the word “Lamp” under the “Obj” section of the window. Press “+” (or “\”) to scroll through the list of possible treasure chests until you reach “Bow” (do not choose “Bow&Arrow” –it won’t display a message when you take it.) Then change the editing mode to “Item” and click on each heart, then using “N”, “M”, “J”, and “K” to switch the item to “Arrows”. Save.

    You Can Change Treasure Chest Contents With + and \

    You Can Change Treasure Chest Contents With + and "

    Link now has access to the bow, 15 arrows, and not much else. Play through your game a bit, and you’ll notice that arrows are a bit hard to come by. Also, we’re stuck in the basement, due to that wall we placed in the last section. We’ll change that next.

    Step 4: A Complete Roomlet

    Modify the following in Entrance 32. Note that the “size” I specify might require you to stretch the component horizontally, vertically, or both.

    Comment Before After
    lights x:1A y:09 size: 00
    lights x:1A y:15 size: 00
    lights x:2F y:25 x:30 y:09
    lights obj:80 x:3B y:11 front
    wall x:2A y:14 x:27 y:14
    wall x:18 y:09 x:25 y:14 size:01
    wall x:2A y:18 x:27 y:18
    cement x:18 y:18
    cement x:28 y:18 x:26 y:18 back
    wall x:2A y:1A x:27 y:1A
    wall x:2D y:17 x:2A y:16 front
    wall x:31 y:17 x:31 y:16 size:00
    stairs obj:F9D x:2E y:16 front
    wall obj:113 x:33 y:16 back
    wall obj:004 x:33 y:17 size:01 back
    water x:30 y:0F x:2E y:0F
    water obj:0C8 x:36 y:0F size:01 back
    edge x:30 y:0F x:2D y:0F size:06 front
    edge x:30 y:0E x:2D y:0E size:08 front
    edge obj:044 x:2D y:16 size:06 front
    edge x:30 y:17 x:35 y:17 size:00

    Here’s what it looks like after you’re finished:

    Our Entry Room to the Castle Basement is Complete

    Our Entry Room to the Castle Basement is Complete

    You’ve learned a lot about from this, I hope. For example, you’ve learned that tiles can overlay like crazy, and the movement rules are determined in a very sensible, visible fashion. (This becomes less intuitive once you start using two layers). You may have also learned that if you left click on an object, then right-click and choose “Add Object”, the default object will be the type of the first object you clicked on. This makes it really easy to insert a new tile if you can see a similar one on-screen. It’s a real time-saver.

    Step 5: Doors and Staircases

    We now move on to one of the most difficult parts of dugeon editing: doors and staircases. Actually, that’s a lie: everything’s amazingly difficult. But doors (and staircases) require actual explaining, not just hex mangling, to really understand the concept. And then there’s that mind-numbing number pad “feature”, which I consider to be pure evil.

    Backup your ROM and open the working copy in Hyrule Magic. (We can switch to Windows for a while, yes?) Now, click on layer 3 in “Edit”, and then right-click and choose “insert a door”. Scroll up to door 000, click on it, and click “Ok”.

    Add a New Door of Type 000

    Add a New Door of Type 000

    Now, left-click on the door you just placed. This is where Hyrule Magic gets evil. You need to press the arrow keys on the number keypad in order to move the door around. The regular arrow keys won’t work. (On a laptop, you’ll have to use the function key, probably). In my opinion, this is a pure sin of interface design —why would you ever assign different functionality to these two sets of keys. Even Ctrl+Arrow is a better solution.

    Press the right arrow until the “Pos” of your new door reads “8”. This may look a bit odd for now, but we’ll fix that later.

    Use the Number Pad's Arrow Keys to Move the Door

    Use the Number Pad's Arrow Keys to Move the Door

    You can also add this door on layer 1. However, if you do that, the door on the left might disappear; you can manually add it back (it’s door type 001, but we’ll change that later.) A bigger problem is what happens if the entrance door disappears. If that is the case, you have to re-add it —and it can only be added to layer 3. Also, there are two doors which must be stacked to enforce the entrance. Do the following:

    • First put down door type 02A and align it over where the exit was.
    • Then, put down door type 033 and move it directly over the door you just placed.
    • Now save. Your entrance functionality should have been restored. If not, make sure that you placed both doors on layer 3.

    Now that you’ve fixed the entrance (or maybe you were lucky and didn’t have to) you should add another door, of type 016, and move it to position 7. Finally, click on the left-most door (the one that was already there) and move it to position 6. Right-click on it, select “choose an object”, and change it to object 000.

    I recommend keeping all doors in layer 3, but it’s up to you. Either