Blog Info: I finally did it: I took on a task that was too big for me to finish. Fortunately, I managed to get its basic functionality all done by press time. If you can follow along, you’ll know more than enough to continue on your own from here.
Quick Note: I’m still trying to fix some of the formatting, but the content’s all here, so I decided just to post it. I’ll re-tab the untabbed code later. 
Back-story: Our previous notifier was pretty hackneyed. It was also heavily coupled with the client program’s source, which might not be such a good idea. (What if the developer released the next version in Haskell or Forth? What if he closed the client source for the next release? What if —and this is the most likely— we don’t want to make our notifier any fancier using Win32 C++?) We need a new solution, and that means it’s time to pull out the big guns.
Goal: Write a new notifier plugin. Make it operate external to the Eternal Lands program. Don’t slow down the client or the server. Don’t break any rules. Allow multiple types of notifications. Make it flashy.
Licensing Notice
This post provides a lot of code. While I normally provide all code in the public domain, I am restricting this post’s code with a license. Scroll to the bottom of the post to read it. This license is not for my benefit, but for the benefit of the EL servers and admins. Basically, it prohibits you from using this code to automate, or to modify it in any way that causes undue strain on the servers. You cannot inject events, nor can you delay sending them between the client and server. Finally, you cannot modify the thread priority of the event handler. The EL team has volunteered their time and creativity for your benefit, and I won’t have my code used to cause trouble for them.
Step 1: General Approach
Most of the problems in our last notifier stemmed from hastiness. In light of that, this section will take it slow and explain why we chose to do things the way we did. We’ll discuss some alternative approaches. But first, a general overview of what we want, and why our last plugin is no longer sufficient:
- Event Notification — We want to see a message when something happens in Eternal Lands. Our initial program tagged harvest completion, but that was all. (Bonus question: How can you stop harvesting without displaying the “You stopped harvesting” message? It’s a tricky one… if you’re not sure, try harvesting some Seridium until you run out of Matter Essences.)
- Notification Timeliness — We want to show notifications only when necessary, and hide them when they’re not needed anymore. The previous plugin showed them if the Eternal Lands window was not the foreground window, and hid them when a new message came in. Later, we found this to be unsatisfactory. Better to hide all messages when the EL window gains focus.
- Variety of Notifications — Seeing a non-descript “You stopped harvesting” is different from seeing a shiny “You found an Enrichment Stone!” or a red “You have died and gone to the underworld”. We’ll add multiple types of notifications, which will necessitate allowing them to stack, fade in, and fade out. If you guessed that we won’t be doing all this graphical work in Win32 C++, you guessed right!
- Separation from Client Code — We won’t be using C++, nor will we be building directly on top of the Eternal Lands source. Rather, we’ll build a separate module that will load the EL client, and then filter all messages sent to it. We also want a system tray icon, and we might even consider removing the Eternal Lands window from the taskbar entirely.
- Fast and Extensible — We want the code to be fast (sorry, Perl). We want it to be easy to add new filters (sorry, C). We want it to operate directly on the network data (sorry, Forth) and have a great graphics library. If it’s native to Windows, all the better. Our choice is C#.
Why not just read the client log file? If you browse to “C:\Users\User Name\Documents\Eternal Lands\main\chat_log.txt” —possibly replacing “main” with the name of the server you plan on, and “Documents” with “My Documents” on Windows 7— you’ll find a nice log of (most of) the chat window. Fiddle a bit more, and you’ll find that it’s updated in real time. Why, then, do we not simply tail this file? In Windows, this is pretty easy to do in Java:
public void run() {
File f = new File("C:\\ ... \\chat_long.txt");
long fPointer = ;/* Seek to the end of the file */
for (;;) {
Thread.sleep(100);
long len = f.length();
if (len < fPointer) //Log was reset
fPointer = len;
else if (len > fPointer) { //File was appended to
RandomAccessFile raf = new RandomAccessFile(f, "r");
raf.seek(fPointer);
String line = null;
while ((line = raf.readLine()) != null)
processMessage(line);
fPointer = raf.getFilePointer();
raf.close();
}
}
}Don’t use this code; it doesn’t check for exceptions, and it isn’t mine (it’s part of the java-tail project). More importantly, it doesn’t really work if the file is being constantly added to. You could re-read the file after exceptions, but then you’re entering into the messy possibility of reading the same line twice.
On a totally different level, this is sub-optimal because the client log doesn’t contain everything. What if you wanted to write a filter that tells you if you’ve been interrupted from harvesting by a PKer instead of a sub-lethal cave-in? Battles don’t send messages to the console, unless you die. Wouldn’t you rather find out while you still have a fighting chance?
Why not use Java? This question applies to any other language, but I feel the most reasonable alternative is Java. (I consider Visual Basic .NET to be identical to C# for all such discussions, by the way.) Java is a very good language, and there is no reason not to use it. Your favorite language is also probably a good choice, although I might not have heard of it. My reason for choosing C# was that, of all the candidates I considered, it was capable of (read: “I knew how to use it to”) solving all the sub-tasks I needed. Feel free to follow along in Python or Ruby or Tcl/Tk, converting the code as you like. Ooh, or Delphi; I like Delphi. Regardless, I’m doing this in C#, and any intermediate programmer should be able to follow along.
Why not patch the client code? My earlier reasons for not simply extending on our earlier project were true, but also somewhat motivation by the fact that I knew I was going to use a much higher-level (and thus incompatible) language from the start. What if you started this project using C++? In that case, I’d still encourage you to keep your code totally separate. Besides all the previously-listed reasons, here’s one more: bugs. If you’re recompiling the entire Eternal Lands code base with each new notification, then you’re bound to slip up from time to time. This is part of development; you can release bug-free software, but your internal builds will be fraught with bugs. Now, given the nature of bugs in C++, this means that your code could easily corrupt some of Entropy’s code. If this results in sending a million messages a second to the server, “ya might get banned”. That said, if your bug is passive, and other people use your plugin, you start to run a real risk of bringing down the server. And even if your bug is harmless, it will still confuse developers trying to nail it down. Trust me when I say that tracking a bug to third-party patches does not make developers happy.
How will we catch messages? We’ll start our program first, which should give it priority over any child processes and their system resources. Then, we’ll connect our program to the Eternal Lands server, and connect the EL Client to our program (by adding a new server to the config file). Finally, as we ferry messages between client and server, we’ll send them off to a separate, low-priority thread where we can match then against an arbitrary number of filters, spawning pop-up windows as we identify messages of interest.
Step 2: Connecting to the client
The first thing we need to do it connect to the server. From the Eternal Lands server config file (D:\Programs\Eternal Lands\servers.lst), we know that the “main” server connects to port 2000 on game.eternal-lands.com. Open up Visual Studio (the free Express Edition for C# is pretty nice) and start a new project. Make it a “Windows Forms Application”. You’ll be presented with the design view for “Form1.cs”. In case you aren’t familiar with Visual Studio, take a minute to note the toolbox (circled in red on the left) and the Solution explorer on the right. Right click on Form1.cs in the solution explorer and note the two options shown circled in blue. “View Designer” will bring up the form designer, like you see now. “View Code” will bring up the code that runs everything behind the scenes. Note that C# supports “partial” classes, so the “View Code” option will not show you any code generated by Visual Studio, unless you are running a very, very old version of Visual Studio.
Visual Studio's "Hiding Tool Bars"
Open the toolbox (by clicking on it) and drag a “Text Box” control onto your Form. (This is under the “Common Controls” group). Now, right-click on this box and choose “Properties”. The most important property is called “(Name)”, because this is what we’ll use to identify the control in our C# code. The following table details which controls to add, and what properties to set for them. All other properties can be left at their defaults, or you can change them to your liking. For example, you might change the “font” for your controls, or the “BorderStyle”. Go for it.
| TextBox* | |
| (Name) | txtPathToELFolder |
| Text | C:\Program Files\Eternal Lands |
*Note: If you installed Eternal Lands to a different location, set the Text property accordingly. Escaped backslashes are not necessary when entering this string in the visual editor.
| Button | |
| (Name) | btnRead |
| Text | Read |
| ComboBox | |
| (Name) | cmbServers |
| Enabled | False |
| Button | |
| (Name) | btnConnect |
| Text | Connect |
| Enabled | False |
| Label | |
| (Name) | lblZero |
| Text | 0 |
| TextAlign | MiddleRight |
| Font | Courier New, 14.25pt, style=Bold |
| ForeColor | DarkRed |
| AutoSize | False |
| Size | 346, 399 |
| TextBox | |
| (Name) | txtConsole |
| ReadOnly | True |
| Multiline | True |
| Size | 306, 188 |
Arrange these components like so:
Line Up Your Components Roughly Like This
A quick note on Hungarian Notation: in general, I do not support this hackneyed way of naming variables, especially in a strongly-typed language like C#. However, for GUI components, I often find myself saying “Ah, what was the name of that button I wanted?” By naming controls in an ordered fashion, I can type btn<Space> and get some help from Visual Studio’s auto-complete feature. Having large numbers of (essentially) global variables for your form controls is unavoidable in C#, and I find this approach scales really well, despite the frown that most people get when they see Hungarian Notation.
Compile your program (Build –> Build Solution) and then run it (Debug –> Start Without Debugging). None of the software logic is hooked up yet, but the general idea is that the user can change the client directory, then click “Read” to read the list of available servers. Next, the user can choose a server to connect to from the combo box, and click “Connect”. Log messages will be shown in our multi-line Text Box control, and the mysterious “Zero Count” will be used for something which will be explained later. Well, let’s start the magic.
Click on the “Read” button, and go to its properties. Notice that little lightning bolt, circled below in red? If you click that, you will switch to “Event” triggers. Click the button left of it to go back to properties view.
Bad Design Puts This Very Important Button in a Very Obscure Place.
Click the event button, find the event called Click, and type “ReadELFiles” in the box. Then, press “Enter”. Visual Studio will bring you to Form1.cs’s Code View, with a skeleton implementation of the ReadELFiles function used for your Click event delegate. Java users will be saying “What?”, as delegates and events make little sense from a minimalist point of view. We’ll have a good example of the power of delegates later.
Here’s the implementation of ReadELFiles:
private void ReadELFiles(object sender, EventArgs e) { //Reload an entirely new list of servers. cmbServers.Items.Clear(); //Don’t load anything if we specified an invalid directory. if (!File.Exists(txtPathToELFolder.Text + "\\" + "el.exe")) { txtConsole.Text = "Error: el.exe doesn't exist in EL folder"; return; } else if (!File.Exists(txtPathToELFolder.Text + "\\" + "servers.lst")) { txtConsole.Text = "Error: servers.lst doesn't exist in EL folder"; return; } //Open our servers file and read it. bool foundLoopback = false; StreamReader srvFile = new StreamReader(txtPathToELFolder.Text + "\\" + "servers.lst"); while (!srvFile.EndOfStream) { String line = srvFile.ReadLine(); if (line == null) break; //Coment? Empty? line = line.Trim(); if (line.StartsWith("#") || line.Length == 0) continue; //Read: //ID, Config_Dir, Address, Port, Description String regex = "([^ \t]+)[ \t]+([^ \t]*)[ \t]+([^ \t]*)[ \t]+([^ \t]*)[ \t]+(.*)"; Match m = Regex.Match(line, regex); ServerInfo sInf = new ServerInfo(); if (m.Success && m.Groups.Count == 6) { sInf.id = m.Groups[1].Value; if (sInf.id.Equals("loopback")) foundLoopback = true; sInf.configDir = m.Groups[2].Value; sInf.address = m.Groups[3].Value; try { sInf.port = Int32.Parse(m.Groups[4].Value); } catch (FormatException) { //Don't add continue; } sInf.description = m.Groups[5].Value; cmbServers.Items.Add(sInf); //Loopback? if (sInf.id.Equals("loopback")) loopbackInfo = sInf; } } srvFile.Close(); //Do we need to add our own local server? if (!foundLoopback) { //Copy existing server file StreamReader inFile = new StreamReader(txtPathToELFolder.Text + "\\" + "servers.lst"); StreamWriter outFile = new StreamWriter(txtPathToELFolder.Text + "\\" + "servers.lst.new"); for (String line = inFile.ReadLine(); line != null; line = inFile.ReadLine()) outFile.WriteLine(line); inFile.Close(); //Add our own category outFile.WriteLine(srvLine); //Close, swap, notify. outFile.Close(); File.Delete(txtPathToELFolder.Text + "\\" + "servers.lst"); File.Move(txtPathToELFolder.Text + "\\" + "servers.lst.new", txtPathToELFolder.Text + "\\" + "servers.lst"); ShowConnectMsg("Created loopback entry."); //Add a new item to the list loopbackInfo = new ServerInfo(); loopbackInfo.id = "loopback"; loopbackInfo.configDir = "main"; loopbackInfo.address = SERVER_ADDR; loopbackInfo.port = SERVER_PORT; cmbServers.Items.Add(loopbackInfo); } if (cmbServers.Items.Count > 0) cmbServers.SelectedIndex = 0; }
Make sure you add the following using directives to the top of Form1.cs, otherwise you’ll get errors stating that “The name <something> does not exist in the current context”. You also won’t get auto-complete functionality until the proper directives are added. (Auto-imports is something I miss dreadfully from Eclipse.)
using System.Text.RegularExpressions; using System.IO;
You will also need the following variables; add them as part of the class (e.g., one line after the opening brace following “public partial class Form1 : Form”)
public const string SERVER_ADDR = "127.0.0.1"; public const string SERVER_PORT = 5421; private string srvLine = "127.0.0.1";"loopback main " + SERVER_ADDR + " " + SERVER_PORT + " Loopback adapter for use with the EL Notifier"; private ServerInfo loopbackInfo;
We also need this “ServerInfo” structure everyone’s been talking about. Add it as part of the namespace (not the class Form1).
public struct ServerInfo { public String id; public String configDir; public String address; public int port; public String description; public override String ToString() { return description; } }
The “ToString()” method is called by the Combo Box control to display the Object it contains; overriding it allows us to dictate that ServerInfo items are displayed by the description. There are better (but less concise) ways of controlling Combo Box output.
Note that structs in C# are very different than structs in C++. Indeed, you could have used a class here; the only difference would have been in its copy semantics, and possibly its memory footprint.
The only thing you’re missing is the ShowConnectMsg delegate, but for now just assume it’s a standard function call that does some non-essential logging. Let’s dissect the code we just wrote!
The SERVER_ADDR and SERVER_PORT variables help us to set up our program as a middleware between the EL Client and the EL Server. Our program has both client and server aspects; these two variables define access to the latter. We use srvLine to shim the servers list so that it can access our “Loopback” interface. Note that srvLine is effectively constant, even though it is not declared that way.
We are basically equivocating on the program’s socket expectations. From Wikipedia:
An Internet socket is characterized by a unique combination of the following:
- Protocol (TCP, UDP or raw IP).
- Local socket address (Local IP address and port number)
- Remote socket address (Only for established TCP sockets.)
The original connection from EL Client to EL Server has one socket: a TCP-enabled connection from 127.0.0.1:???? to game.eternal-lands.com:2000. This socket is bi-directional. We don’t know the local port, because local ports are almost always randomly assigned. We don’t technically know the server port either, but 2000 is what we connect to, so I’ll list that here.
Our shimmed middleware creates two bi-directional sockets:
TCP, 127.0.0.1:???? ? SERVER_ADDR:SERVER_PORT
TCP, SERVER_ADDR:???? ? game.eternal-lands.com:2000
Note that, as per the usual, we don’t know the local ports. Either way, we are guaranteed that they will be unique.
All that explanation for just three lines of code? Well, if you’re a non-network guy like me, figuring this out is one of the hardest parts of this project. Now, on to the rest of the code:
The ReadELFiles function is pretty easy to follow:
- Note that the function parameters “sender” and “e” are not used in our function. These are mandated by the .NET platform, but are not needed for our simple example.
- The foundLoopback variable is used to check each line read from servers.lst. If the id of that line is “loopback”, then we know a loopback entry exists. If, at the end of scanning, no such entry was found, we just add it to the end of the file. That way, when we start Eternal Lands, we can connect to the loopback server (instead of main or pk or testing) and fool the client into thinking it’s connecting to the real server, instead of our fake, localhost middleware.
- Note that we should check the loopback entry to ensure that its address and port parameters match ours. I’ll leave this as an exercise in regular expressions.
- Some regex pattern matching is used to filter each entry in our servers.lst file. We use regexes to check if a line matches a specific pattern, and we then use the Groups property to extract what was matched. If you don’t understand pattern matching, just skip the details. Learn it later, of course —it’s a fantastic way of dealing with text in almost any modern language.
We will now add the final missing piece of the puzzle. Add the following to the namespace (not the Form1 class):
public delegate void ShowConnectMsgDel(String msg);
Then, add the following function definition to the Form1 class:
private void ShowConnectMsg(String msg) { Control ctl = txtConsole; if (ctl.InvokeRequired) ctl.Invoke(new ShowConnectMsgDel(ShowConnectMsg), msg); else ctl.Text = msg; }
Our program will now compile. (I’ll explain delegates later, when it makes more sense to.) Run it, click the “Read” button, and you will see “Main game server” appear in the still-disabled Combo Box.
A Properly-Formatted Servers File Is Parsed.
If your program doesn’t compile, have a look at Listing 1 and try to figure out where you went wrong. (Apologies for the PDF; WordPress doesn’t like .txt files.) If it doesn’t load the server list, make sure you supplied the proper path to the EL Client folder.
Now, add the following code to the end of your ReadELFiles function:
//Enable the remainder of our form cmbServers.Enabled = true; btnConnect.Enabled = true;
That should enable everything you need. Now, in Design mode, click on the “Connect” button and give it a click Event called ConnectToServer. You’ll be given a function skeleton again; here’s what we’ll add to it:
private void ConnectToServer(object sender, EventArgs e) { //Connect to our server UpdateConsole("Connecting....."); Thread t = new Thread(new ThreadStart(Check_Server)); t.IsBackground = true; t.Start(); }
This requires the CheckServers function, which will run in a separate thread:
private void Check_Server() { //Our middleware acts as a client in this case. TcpClient tcpclnt = new TcpClient(); Stream srvStream; //Get our currently selected server and start it ServerInfo sInfo = (ServerInfo)cmbServers.SelectedItem; tcpclnt.Connect(sInfo.address, sInfo.port); //Display any messages we receive. srvStream = tcpclnt.GetStream(); byte[] msg = new byte[1024]; bool done = false; for (; !done; ) { int amtRead; try { amtRead = srvStream.Read(msg, 0, msg.Length); } catch (IOException) { UpdateConsole("\r\nIOException; closing Server"); amtRead = 0; } if (amtRead != 0) { //Echo to console String line = "MSG: ["; for (int i=0; i<amtRead; i++) { byte b = msg[i]; if (b >= 32 && b <= 126) line += (char)b; else line += '.'; } line += "]"; UpdateConsole("\r\n"+line); } else { UpdateConsole("\r\nRead zero; closing Server"); done = true; } } //Must close BOTH srvStream.Close(); tcpclnt.Close(); //Ready to go again? UpdateConsole("\r\nServer Stopped"); }
Make sure you add the following using directives to the top of your file:
using System.Threading; using System.Net.Sockets;
After that, you’ll only get errors for the UpdateConsole function, which we’ll implement now:
private void UpdateConsole(String msg) { Control ctl = txtConsole; if (ctl.InvokeRequired) ctl.Invoke(new AddToConsoleDel(UpdateConsole), msg); else ctl.Text += msg; }
Add the following to our namespace, and we’ll be done:
public delegate void AddToConsoleDel(String msg);
Run the program, click “Read” to get a server list, then choose the “Main game server” from your dropdown list and hit “Connect”. You should see a nascent echo of the regular log-in message, and some unreadable garbage.
The "Garbage" Is Actually the Important Information, As We'll See Later.
If this just doesn’t work, have a look at Listing 2 to see where you went wrong.
If you scan the code, you’ll find everything in order. First, we create a TCPClient object to manage our TCP connection to the server. We then Connect it to the server, based on the item chosen in the drop-down list. Finally, we continue to read up to 1kb at a time from the server. To the best of my knowledge, the Eternal Lands server never sends more than a few hundred bytes of data at once, so we don’t have to merge messages.
Note that this all has to happen in a different thread, to avoid throttling the GUI dispatch thread. If you just ran a tight loop in the main Thread, your Windows Form would become sluggish and unresponsive. The C#-ism:
Thread t = new Thread(new ThreadStart(Function_Name));
…capitalizes on delegates to encapsulate a function (Function_Name) in a new thread. This is a reasonably good example of a delegate, but we have a much better one: our UpdateConsole function.
The problem with ThreadStart is that it’s too easy to guess what it does, without actually understanding why we choose to do it this way. Instead, let’s have a look at UpdateConsole. I’ve highlighted it differently below:
private void UpdateConsole(String msg)
{
Control ctl = txtConsole;
if (ctl.InvokeRequired)
ctl.Invoke(new AddToConsoleDel(UpdateConsole), msg);
else
ctl.Text += msg;
}
First, our variables are declared in red. One is the value we wish to update the console with (as a String) and the other is a local variable used to reference the Text Box itself. Since all Text Boxes subclass Control, we can store this variable in a generic fashion. Note that the variable ctl was created to emphasize that this technique can be used on any part of our form. We could have put ctl as a second function variable, if we had multiple consoles.
Next, the blue text. The “InvokeRequired” property (think of it like a read-only variable) is a nifty shorthand which tells us if the given control was created by the thread we are currently running. As all you GUI programmers know, most actions that update GUI components should be done in the master dispatch loop of the thread that created the GUI. If, for example, this control was created in a random thread which is now finished running, then updating it from another thread will cause all sorts of exceptions at run-time. We could dispatch a new thread for each update operation, but the InvokeRequired property helps us to avoid spawning threads when we don’t need to.
The purple text is run if an Invoke is Required. It calls the Invoke method of ctl, which adds a new event to a thread guaranteed to be valid for modifying ctl. This thread is of type AddToConsoleDel, which we defined earlier as:
delegate void AddToConsoleDel(String msg)
…in other words, as a delegated function with one argument that returns nothing (void). This matches the function signature of our UpdateConsole function. Look back at the purple text, and you’ll see that we call new AddToConsoleDel(UpdateConsole) to create a new delegate of type AddToConsoleDel that will run the UpdateConsole function when called. The second parameter to Invoke is the argument set that we intend to pass to our AddToConsoleDel delegate. Since UpdateConsole only takes one argument, we only pass in one (msg).
Which leads us to the green text: what happens if we are running in the right thread. If this is the case, we simply append the message string to the Text Box’s existing Text property.
So, the logic of our function is simple. If the thread is capable of modifying the control directly, it just appends the text. Otherwise, it creates a new delegate to the same function, passes along the same method arguments, and asks the given control to Invoke this delegate. That ensures that next time the function is called (by the delegate this time) it has the necessary permission to modify the control. This logic is a bit tricky to catch the first time you see it, but it’s one of the main design patterns of C#. And, it has the benefit of keeping the actual logic of the method in the same place as the invoke logic of the delegate. So we can just call UpdateConsole and not have to care about whether a thread is spun off to do the dirty work.
You can think of a delegate as a pointer to a function, with enhanced type-safety, if that’s any easier for you.
Take a break, grab a coffee, you deserve a break.
Step 3: Passing Messages Along Properly
Our middleware can locate the Eternal Lands server, and it establishes a TCP connection (proven by the fact that it receives two messages in sequence, not just two copies of the same message). We also jumped the gun and added a loopback entry to the client’s list of servers, to allow us to force the client to connect to our program instead of the real server. Now, let’s put that new entry to work, and tie the client and server together so that you can play Eternal Lands with our program as a local proxy.
Open your project in Visual Studio, and right-click on the “Solution” (top-most item) in the “Solution Explorer”. Choose “Add”, then “New Project”, then select “Class Library” and name it Eternal Lands Connector. Press “Ok”. Now, right-click on your main project (“Windows Forms Application 1”, in my case) and choose “Project Dependencies”. In the “Depends on” box, check the “Eternal Lands Connector” project. Click “Ok”.
The purpose of this secondary project is to isolate our code. Once we’ve written the C# necessary to tunnel messages from the client to the server, we’ll then be adding a whole bunch of filtering and display code. We don’t want to accidentally break our message passing code, so we’ll keep in a totally separate project. You could manage this project separately, and manually add the DLL to the main Windows Forms project each time you update it, but Visual Studio’s dependency tracker allows you to do this automatically.
Let’s get started. First, rename Class1.cs to ELMiddleware.cs. We’ll need another file, though (Right-click “Eternal Lands Connector”, choose “Add”, then “New Item…”, then select “Class”) called SleepQueue.cs. This queue manages a list of messages, and stops processing them when the queue is empty. It sets itself to sleep until a new message is received. This signaling approach is far more efficient than constantly checking if the queue has items (spinning). Here’s the code for this class:
using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading; using System.Net; using System.Net.Sockets; using System.IO; namespace Eternal_Lands_Connector { public abstract class SleepQueue : IDisposable { //For threading: allow sleeping when no messages are received, by using an EventWaitHandle private Thread t; private EventWaitHandle wHandle = new AutoResetEvent(false); private Queue<PacketData> waitingMessages = new Queue<PacketData>(); //Add a packet to the list of waiting messages. // Wake the main thread. public virtual void AddPacket(PacketData p) { lock (waitingMessages) waitingMessages.Enqueue(p); wHandle.Set(); } //What priority should the message queue thread be? // Sub-classes can override this to balance performance. public virtual ThreadPriority getInitialPriority() { return ThreadPriority.Normal; } //Dispose properly public void Dispose() { // Signal the consumer to exit. AddPacket(null); if (t != null) t.Join(); if (wHandle != null) wHandle.Close(); } //Start the queueing process. public void startQueueing() { t = new Thread(new ThreadStart(Handle_Queue)); t.IsBackground = true; t.Priority = getInitialPriority(); t.Start(); } private void Handle_Queue() { //Do pre-inits Pre_Init_Activity(); //Listen for queue updates PacketData msg = null; for (; this.isValid(); ) { //Get the next task msg = null; lock(waitingMessages) { if (waitingMessages.Count > 0) { msg = waitingMessages.Dequeue(); if (msg == null) { //Done. this.On_Message(msg); return; } } } //A bit confusing to have two null checks, but it makes sense. if (msg == null) { //No more tasks - wait for a signal wHandle.WaitOne(); } else { this.On_Message(msg); } } this.On_Thread_Close(); } // //Un-implemented functionality // //When does this queue exit? protected abstract bool isValid(); //What should we do with each message? protected abstract void On_Message(PacketData msg); //What should happen when the thread closes? protected abstract void On_Thread_Close(); //What should happen before the message loop begins? (After the thread is started) protected abstract void Pre_Init_Activity(); } }
This depends on the PacketData.cs file, which is simply a wrapper on a byte array coupled with an id character:
using System; using System.Collections.Generic; using System.Linq; using System.Text; namespace Eternal_Lands_Connector { public class PacketData { public PacketData(byte[] data, char from) { this.data = data; this.from = from; } public byte[] data; public char from; } }
The technical details of the Sleep Queue are not important, so long as you realize that:
- Every message posted to the queue gets processed.
- You can post null to stop processing and end the thread.
- The thread takes up no CPU time when there are no messages waiting.
- Sub-classes will have to tell the queue when to stop processing (naturally), what to do with messages that are received, and what to do directly before entering and after leaving the message loop.
We now have everything we need to implement the ELMiddleware class, whose purpose is to handle all messages between the client and the server.
using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading; using System.Net; using System.Net.Sockets; using System.IO; namespace Eternal_Lands_Connector { //Global public delegate void AddPacketDel(PacketData msg); public delegate void ShowConnectMsgDel(String msg); public delegate void AddToConsoleDel(String msg); public delegate void IncrPacketCountDel(int amt); public struct ServerInfo { public String id; public String configDir; public String address; public int port; public String description; public override String ToString() { return description; } } public class ELMiddleware : SleepQueue { //Hooks for controlling the client and server. private ELClientHook client; private ELServerHook server; //Saved private ServerInfo currServer; private ServerInfo loopback; private String elClientDir; private ShowConnectMsgDel ShowConnect; private AddToConsoleDel UpdateConsole; private IncrPacketCountDel IncrPacketCount; private int hookThreadID; public override void AddPacket(PacketData p) { //Log IncrPacketCount(1); //Count base.AddPacket(p); } private bool started = false; public bool Started { get { return started; } } public ELMiddleware(ELClientHook client, ELServerHook server) { //Save this.client = client; this.server = server; //Hook this.client.PacketHook = new AddPacketDel(AddPacket); this.server.PacketHook = new AddPacketDel(AddPacket); } public void startAll(ServerInfo currServer, ServerInfo loopback, String elClientDir, ShowConnectMsgDel ShowConnect, AddToConsoleDel UpdateConsole, IncrPacketCountDel IncrPacketCount, int hookThreadID) { //Don't start twice if (started || client.Started || server.Started) return; started = true; //Save this.currServer = currServer; this.loopback = loopback; this.elClientDir = elClientDir; this.ShowConnect = ShowConnect; this.UpdateConsole = UpdateConsole; this.IncrPacketCount = IncrPacketCount; this.hookThreadID = hookThreadID; //Start the sleep queues base.startQueueing(); } protected override void Pre_Init_Activity() { //Init Log ShowConnect("EL Notifier Init"); //Start the server server.hookServer(currServer, UpdateConsole); //Start the client client.hookClient(loopback, elClientDir, UpdateConsole, hookThreadID); //Spin-wait for both while (!server.ReadyForData || !client.ReadyForData) Thread.Sleep(5); UpdateConsole("\r\nSpin-waiting done"); } protected override void On_Message(PacketData msg) { //Count IncrPacketCount(-1); if (msg == null) return; //Forward this message to the appropriate third party if (msg.from == 'C') { //Send from the client to the server. server.sendData(msg.data); } else if (msg.from == 'S') { //Send from the server to the client client.sendData(msg.data); } else UpdateConsole("\r\nBad message code: " + msg.from); } protected override bool isValid() { return client.Started && server.Started; } protected override void On_Thread_Close() { //Make sure everything closed nicely if (!server.Done) server.Done = true; if (!client.Done) client.Done = true; //Done started = false; } } }
Add two more classes to the project, called ELClientHook.cs and ELServerHook.cs, respectively. Mark each class public, but don’t add any implementation yet.
Let’s review the code for ELMiddleware. At the top, we have some global delegate definitions, presumably used for logging. We also have our ServerInfo struct —if you think this means we’ll be removing a lot of code from Form1.cs then you guessed right. Our ELMiddleware class extends the SleepQueue abstract class. It contains several delegate references, a mysterious hookThreadID, and a reference to our client and server hooks.
Most of the code after this is obvious, and should give you an idea what kind of functionality we’ll need in ELClientHook and ELServerHook. The onMessage() function simply checks the sender of the message, and then ferries the message across to the recipient. We will later add the ability to react to ‘S’ class messages.
Our client and server hooks will need to be able to “take” messages from the middleware, in addition to other miscellaneous features. First, let’s add ELServerHook. Its functionality will be very similar to the previous chapter’s exercises.
using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Net; using System.IO; using System.Net.Sockets; using System.Threading; namespace Eternal_Lands_Connector { public class ELServerHook { private TcpClient tcpclnt; private Stream srvStream; private ServerInfo srv; //Callbacks private AddToConsoleDel UpdateConsole; private bool started = false; public bool Started { get { return started; } } private bool done; public bool Done { get { return done; } set { done = value; } } private bool readyForData = false; public bool ReadyForData { get { return readyForData; } } private AddPacketDel packetDel; public AddPacketDel PacketHook { get { return this.packetDel; } set { this.packetDel = value; } } public void hookServer(ServerInfo srv, AddToConsoleDel UpdateConsole) { //Don't start twice if (started) return; started = true; //Save this.srv = srv; this.UpdateConsole = UpdateConsole; //Start tcpclnt = new TcpClient(); UpdateConsole("Connecting....."); Thread t = new Thread(new ThreadStart(Check_Server)); t.IsBackground = true; t.Start(); } public void sendData(byte[] data) { if (!readyForData) throw new Exception("Server is not ready for messsages"); //Send the dat down this stream. Technically, this occurs in the middleware's thread, // so it won't conflict with any data we're reading at the same time. try { srvStream.Write(data, 0, data.Length); } catch (Exception ex) { UpdateConsole("\r\nException writing to server: \r\n " + ex.ToString()); } } private void Check_Server() { //Connect to the server's TCP port. tcpclnt.Connect(srv.address, srv.port); readyForData = true; UpdateConsole("Connected"); //Try reading a bit srvStream = tcpclnt.GetStream(); byte[] msg = new byte[1024]; done = false; for (; !done; ) { int amtRead; try { amtRead = srvStream.Read(msg, 0, msg.Length); } catch (IOException) { UpdateConsole("\r\nIOException; closing Server"); amtRead = 0; } if (amtRead != 0) { //Make a new packet, add it to the middleware's queue byte[] pk = new byte[amtRead]; Array.Copy(msg, pk, amtRead); PacketHook(new PacketData(pk, 'S')); } else { UpdateConsole("\r\nRead zero; closing Server"); done = true; } } //Must close BOTH srvStream.Close(); tcpclnt.Close(); //Ready to go again? UpdateConsole("\r\nServer Stopped"); Console.WriteLine("Server hook stopped"); started = false; } } }
The implementation is almost exactly what we had before. The call to srvStream.Read() is blocking, by the way —no Sleep Queue is necessary.
The ELClientHook implementation is much more elusive:
using System; using System.Collections.Generic; using System.Linq; using System.Net; using System.IO; using System.Net.Sockets; using System.Text; using System.Text.RegularExpressions; using System.Diagnostics; using System.Threading; using System.Windows.Forms; using System.Runtime.InteropServices; namespace Eternal_Lands_Connector { //Just a helper class; no threads. public class ELClientHook { //Const public const string SERVER_ADDR = "127.0.0.1"; public const int SERVER_PORT = 5421; private ServerInfo loopback; private String elClientDir; private TcpListener listener; private Stream clientStr; private Form mainForm; private int hookThreadID; private bool elDone = false; private bool started = false; public bool Started { get { return started; } } private bool done; public bool Done { get { return done; } set { done = value; } } private bool readyForData = false; public bool ReadyForData { get { return readyForData; } } private AddPacketDel packetDel; public AddPacketDel PacketHook { get { return this.packetDel; } set { this.packetDel = value; } } //Callbacks private AddToConsoleDel UpdateConsole; public void hookClient(ServerInfo loopback, String elClientDir, AddToConsoleDel UpdateConsole, int hookThreadID) { //Don't start twice if (started) return; started = true; //Save this.loopback = loopback; this.UpdateConsole = UpdateConsole; this.elClientDir = elClientDir; this.hookThreadID = hookThreadID; //Start background client Thread t = new Thread(new ThreadStart(Start_Client)); t.IsBackground = true; t.Start(); } public void sendData(byte[] data) { if (!readyForData) throw new Exception("Server is not ready for messsages"); //Send the dat down this stream. Technically, this occurs in the middleware's thread, // so it won't conflict with any data we're reading at the time. try { clientStr.Write(data, 0, data.Length); //Logging code - uncomment for development /*StringBuilder sb = new StringBuilder(); sb.Append("\r\nClient Write: ["); foreach (byte b in data) sb.Append(Convert.ToString(b, 16).PadLeft(2, '0') + ":"); sb.Append("]"); UpdateConsole(sb.ToString());*/ } catch (Exception ex) { UpdateConsole("\r\nException writing to client: \r\n " + ex.ToString()); } } private void EL_Stopped(Object sender, EventArgs e) { elDone = true; } //Get our handle (no idea why this requires an invoke... maybe it creates it?) private delegate IntPtr GetFormHandleDel(Form f); private IntPtr GetFormHandle(Form f) { if (f.InvokeRequired) return (IntPtr)f.Invoke(new GetFormHandleDel(GetFormHandle), f); else return f.Handle; } private bool findMainForm(int[] allowedProcessIDs, String matcherRegex, int hookThreadID) { return true; } private void EL_Window_Activate() { Console.WriteLine("EL Client Window Activated"); } private void EL_Window_Deactivate() { Console.WriteLine("EL Client Window Deactivated"); } private void Start_Client() { //Start the socket listener = new TcpListener(IPAddress.Parse(SERVER_ADDR), SERVER_PORT); listener.Start(); UpdateConsole("\r\nLoopback server running at: " + listener.LocalEndpoint); //Start the EL client program with the server specified as "loopback" UpdateConsole("\r\nStarting EL Client..."); ProcessStartInfo ps = new ProcessStartInfo(); ps.FileName = elClientDir + "\\el.exe"; ps.Arguments = loopback.id; ps.WorkingDirectory = elClientDir; ps.ErrorDialog = true; Process p = Process.Start(ps); elDone = false; p.Exited += new EventHandler(EL_Stopped); //Get the full-screen window for this application mainForm = null; while (mainForm == null) { int[] validIDs = new int[p.Threads.Count]; { int id = 0; foreach (ProcessThread t in p.Threads) validIDs[id++] = t.Id; } if (findMainForm(validIDs, "Eternal Lands", hookThreadID)) break; Thread.Sleep(5); } UpdateConsole("\r\nClient spin-waiting done"); //Wait for the client to try to connect to us TcpClient socket = listener.AcceptTcpClient(); clientStr = socket.GetStream(); readyForData = true; UpdateConsole("\r\nEL Client tried to connect to us: " + socket.Client.RemoteEndPoint); //Continually read from the client. This is the main client update thread. byte[] msg = new byte[1024]; done = false; for (; !done; ) { //Is the client still running? if (elDone) { UpdateConsole("\r\nGame closed; closing Client"); done = true; } else { int amtRead; try { amtRead = clientStr.Read(msg, 0, msg.Length); } catch (IOException) { UpdateConsole("\r\nIOException; closing Client"); amtRead = 0; } if (amtRead != 0) { //Make a new packet, add it to the middleware's queue byte[] pk = new byte[amtRead]; Array.Copy(msg, pk, amtRead); PacketHook(new PacketData(pk, 'C')); } else { //We're done UpdateConsole("\r\nRead zero; closing Client"); done = true; } } } //Have to close all three clientStr.Close(); socket.Close(); listener.Stop(); //Ready to go again? UpdateConsole("\r\nClient Stopped"); Console.WriteLine("Client hook stopped"); started = false; } } }
You might get an error with the System.Windows.Forms using directive; simply right-click on the “References” section of your Connector’s Project file and choose “Add Reference”. Then, in the “.NET” tab, browse down to “System.Windows.Forms” and click “Ok”. By default, a library project does not have access to the Forms dll, for reasons that remain a mystery to me.
A quick browse through this code reveals that our fake middleware server will be situated at 127.0.0.1, on port 5421. The sendData() function is similar to the server’s copy; however, I’ve included some logging code for your convenience. If you uncomment this, it will print a properly-formatted hexadecimal representation of each message sent from the server to the client. This will allow you to manually debug unknown or tricky messages, and to determine how you should react to them.
The findMainForm() function is currently empty; it will later be used to determine which window is actually hosting the EL client. The only other function of interest is the StartClient() function. Note that we use a ProcessStartInfo object to handle starting a process:
1.We have to set the working directory to the Eternal Lands folder, otherwise the process won’t be able to find its resources (images, sounds, etc.).
2.We start the process el.exe. This, of course, is Windows-specific.
3.We start this process with the argument loopback. In other words, we call “el.exe loopback” from the command line, allowing us to choose which server to connect to.
The Thread-finding code which follows this is messy; just ignore it. Basically, our middleware program will have multiple running threads, and we need to figure out which one is hosting the EL Client’s window. That way, we can detect when it has focus, and avoid showing messages in that case.
Note that, like the server, the client continually reads from the middleware until no more data is sent. You’ll have to change Form1’s code to actually use this new middleware, of course.
using System; using System.Collections.Generic; using System.ComponentModel; using System.Data; using System.Drawing; using System.Linq; using System.Text; using System.Windows.Forms; using Eternal_Lands_Connector; using System.Text.RegularExpressions; using System.IO; using System.Threading; using System.Net.Sockets; namespace WindowsFormsApplication1 { public partial class Form1 : Form { //Count leftover packets. A good first-warning if something goes wrong. private int packetCount; //Our hooks private ELServerHook serverHook; private ELClientHook clientHook; private ELMiddleware middleWare; //Also... private ServerInfo loopbackInfo; private string srvLine = "loopback main " + ELClientHook.SERVER_ADDR + " " + ELClientHook.SERVER_PORT + " Loopback adapter for use with the EL Notifier"; public Form1() { InitializeComponent(); //Initialize connectors. serverHook = new ELServerHook(); clientHook = new ELClientHook(); middleWare = new ELMiddleware(clientHook, serverHook); } private void ReadELFiles(object sender, EventArgs e) { //Reload an entirely new list of servers. cmbServers.Items.Clear(); //Don’t load anything if we specified an invalid directory. if (!File.Exists(txtPathToELFolder.Text + "\\" + "el.exe")) { txtConsole.Text = "Error: el.exe doesn't exist in EL folder"; return; } else if (!File.Exists(txtPathToELFolder.Text + "\\" + "servers.lst")) { txtConsole.Text = "Error: servers.lst doesn't exist in EL folder"; return; } //Open our servers file and read it. bool foundLoopback = false; StreamReader srvFile = new StreamReader(txtPathToELFolder.Text + "\\" + "servers.lst"); while (!srvFile.EndOfStream) { String line = srvFile.ReadLine(); if (line == null) break; //Coment? Empty? line = line.Trim(); if (line.StartsWith("#") || line.Length == 0) continue; //Read: //ID, Config_Dir, Address, Port, Description String regex = "([^ \t]+)[ \t]+([^ \t]*)[ \t]+([^ \t]*)[ \t]+([^ \t]*)[ \t]+(.*)"; Match m = Regex.Match(line, regex); ServerInfo sInf = new ServerInfo(); if (m.Success && m.Groups.Count == 6) { sInf.id = m.Groups[1].Value; if (sInf.id.Equals("loopback")) foundLoopback = true; sInf.configDir = m.Groups[2].Value; sInf.address = m.Groups[3].Value; try { sInf.port = Int32.Parse(m.Groups[4].Value); } catch (FormatException) { //Don't add continue; } sInf.description = m.Groups[5].Value; cmbServers.Items.Add(sInf); //Loopback? if (sInf.id.Equals("loopback")) loopbackInfo = sInf; } } srvFile.Close(); //Do we need to add our own local server? if (!foundLoopback) { //Copy existing server file StreamReader inFile = new StreamReader(txtPathToELFolder.Text + "\\" + "servers.lst"); StreamWriter outFile = new StreamWriter(txtPathToELFolder.Text + "\\" + "servers.lst.new"); for (String line = inFile.ReadLine(); line != null; line = inFile.ReadLine()) outFile.WriteLine(line); inFile.Close(); //Add our own category outFile.WriteLine(srvLine); //Close, swap, notify. outFile.Close(); File.Delete(txtPathToELFolder.Text + "\\" + "servers.lst"); File.Move(txtPathToELFolder.Text + "\\" + "servers.lst.new", txtPathToELFolder.Text + "\\" + "servers.lst"); ShowConnectMsg("Created loopback entry."); //Add a new item to the list loopbackInfo = new ServerInfo(); loopbackInfo.id = "loopback"; loopbackInfo.configDir = "main"; loopbackInfo.address = ELClientHook.SERVER_ADDR; loopbackInfo.port = ELClientHook.SERVER_PORT; cmbServers.Items.Add(loopbackInfo); } if (cmbServers.Items.Count > 0) cmbServers.SelectedIndex = 0; //Enable the remainder of our form cmbServers.Enabled = true; btnConnect.Enabled = true; } private void ConnectToServer(object sender, EventArgs e) { if (middleWare.Started || clientHook.Started || serverHook.Started) return; //Count packetCount = 0; //Get our currently-selected server and start it. ServerInfo srv = (ServerInfo)cmbServers.SelectedItem; middleWare.startAll(srv, loopbackInfo, txtPathToELFolder.Text, new ShowConnectMsgDel(ShowConnectMsg), new AddToConsoleDel(UpdateConsole), new IncrPacketCountDel(IncrPacketCount), Thread.CurrentThread.ManagedThreadId); } //Delegates and implementation details private void ShowConnectMsg(String msg) { Control ctl = txtConsole; if (ctl.InvokeRequired) ctl.Invoke(new ShowConnectMsgDel(ShowConnectMsg), msg); else ctl.Text = msg; } private void UpdateConsole(String msg) { Control ctl = txtConsole; if (ctl.InvokeRequired) ctl.Invoke(new AddToConsoleDel(UpdateConsole), msg); else ctl.Text += msg; } private void IncrPacketCount(int amt) { packetCount += amt; SetText(lblZero, packetCount + ""); } private delegate void SetTextDel(Control ctl, String msg); private void SetText(Control ctl, String msg) { if (ctl.InvokeRequired) ctl.Invoke(new SetTextDel(SetText), ctl, msg); else ctl.Text = msg; } } }
Right-click on the References section in your main Forms project and choose “Add Reference”. Then, in the “Projects” tab, choose “Eternal Lands Connector” and hit “Ok”. This will allow you to use our class library.
The source code is not too complex, and the effect is easy enough to describe: we simply intercept and then deliver all messages from the client to the server, and from the server to the client. At the moment, this is essentially a whole lot of work for nothing. However, now that all messages are passing through our program (our source code, you might say), you can easily tag messages of a specific type, and prompt the user after further processing them. So, this was all “legwork”.
Our middleware in action, connecting to the PK server.
Congratulations, we have now bamboozled both end points. In the next section, we’ll consider how to filter messages without requiring a re-compile of the Connector each time we change our minds about which messages to filter. We’ll also get you started processing Eternal Lands’s images to brighten up our notifier.
Step 4: Filtering Messages (Outside the Core)
At this point, you should have a pretty good idea how to intercept messages of a certain type. Before you dash too far into it, though, may I remind you that one of the reasons we used C# in the first place was to avoid bugs that corrupted our communication layer. We should add the following to the communication core:
- Allow our main Forms class to “register” any number of listeners with the core.
- Before sending a message (to the client), check if it matches any of the listeners’ patterns. If it does, send a copy of the message to the matched listener.
- Ensure that the core runs in a higher-priority thread than the listeners’ implementations. According to the rules of Windows thread priorities, this ensures that the listeners cannot starve the core for cycles.
This way, no amount of chicanery by the listeners can cause the core to malfunction. (And remember, malfunctions are BAD, since we’re not the ones paying for the EL servers.)
Open up ELMiddleware.cs, and add the following member variable to the class:
//An additional matcher
private MessageMatcher msgMatch;
We’ll define MessageMatcher later; for now, scroll down to the definition of startAll, and change it to read:
public void startAll(ServerInfo currServer, MessageMatcher msgMatch, ServerInfo loopback, String elClientDir, ShowConnectMsgDel ShowConnect, AddToConsoleDel UpdateConsole, IncrPacketCountDel IncrPacketCount, int hookThreadID)
Next, somewhere in the body of startAll, save the MessageMatcher that we passed in to the function:
this.msgMatch = msgMatch;
Make sure you start it too; after “base.startQueueing()” add:
msgMatch.startQueueing();
And finally, scroll down until you find the line:
client.sendData(msg.data);
…and add the following underneath it:
//Send the data to an additional thread for logging, parsing, etc.
msgMatch.AddPacket(msg);
That code should speak for itself; our MessageMatcher class will centralize message processing, and the AddPacket method is called for each server message. We do not filter client messages, because it seems that these originate mostly from the user’s actions (which are always known by the user –and must be confirmed by the server, regardless.)
Here, then, is the definition of the MessageMatcher class. Add it to your project (to the Eternal_Lands_Connector project, that is).
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Net;
using System.Net.Sockets;
using System.IO;
namespace Eternal_Lands_Connector
{
public class MessageMatcher : SleepQueue
{
/// <summary>
/// The core of our message matcher
/// </summary>
private Dictionary<Int32, List<MessageHandler>> handlers = new Dictionary<Int32, List<MessageHandler>>();
public void addHandler(MessageHandler h)
{
if (!handlers.ContainsKey(h.getProtocol()))
handlers[h.getProtocol()] = new List<MessageHandler>();
handlers[h.getProtocol()].Add(h);
Console.WriteLine(“Dictionary size: “ + handlers.Count + ” entries[" + h.getProtocol() + "] –> “ + handlers[h.getProtocol()].Count);
}
public AddToConsoleDel UpdateConsole;
protected override void On_Message(PacketData msg)
{
if (msg == null)
return;
//Now, segment into multiple messages
for (int currID = 0; currID < msg.data.Length; )
{
int protocol = msg.data[currID++];
int size = msg.data[currID++] | (msg.data[currID++] << 8);
//Make a new data array
int count = size – 1;
byte[] b = new byte[count];
Array.Copy(msg.data, currID, b, 0, count);
currID += count;
//Match the first relevant handler
if (handlers.ContainsKey(protocol))
{
foreach (MessageHandler h in handlers[protocol])
{
if (h.performMatch(protocol, b, new Dictionary<string, object>()))
{
break;
}
}
}
}
}
/// <summary>
/// Somwhat less-interesting features
/// </summary>
private bool valid = true;
public bool IsValid
{
get
{
return valid;
}
set
{
valid = value;
}
}
//We make this its own class so we can change its priority.
public override System.Threading.ThreadPriority getInitialPriority()
{
return ThreadPriority.BelowNormal;
}
protected override bool isValid()
{
return valid;
}
protected override void On_Thread_Close()
{
//Nothing to do here
}
protected override void Pre_Init_Activity()
{
//Nothing to do here
}
}
}
The MessageHandler class hasn’t been created yet, but let’s just say that it matches all messages first on a “protocol” and then on a more complex “performMatch” query. All of these MessageHandlers are stored indexed by protocol in a hash table. This ensures that we can add lots of processing handlers without slowing down the pattern matching.
Finally, note that MessageMatcher is a SleepQueue, which means that even if we do add far too many handlers of the same protocol (say, one for each buddy in our buddy list?), only our matching will be delayed –the core won’t be.
You might be wondering about the structure of Eternal Lands’s messages. From the processing loop in MessageMatcher, you can tell that every message contains a protocol byte as the first item in the message. The next two bytes describe the size of the remaining data. This is more important than you might think: Eternal Lands often sends multiple messages bunched together, and you can only tell where one ends and the next begins by processing the size bytes. Anything after the first three bytes is dependant on the protocol of the message. Also, for some unknowable reason, the actual size of the remaining data is size-1.
Now, make a class called MessageHandler, in the Eternal_Lands_Connector project:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace Eternal_Lands_Connector
{
public abstract class MessageHandler
{
//All message handlers match on a protocol
public abstract int getProtocol();
//Within a given protocol, does this handler match?
// This should also perform an action
// Decorations used to avoid multiple calculations
public abstract bool performMatch(int protocol, byte[] data, Dictionary<String, Object> decorations);
}
}
Were you to compile your project, you would notice that Form1’s call of middleWare.startAll() is missing an argument: MessageMatcher. Before we go passing in an empty class, let’s try thinking about the types of MessageHandlers we might need. Well, the most basic type will simply match based on the protocol, and defer further processing till later. Then, we might want to match text messages sent from the server, using such things as the color of the text, the channel it was sent to, and a regular expression of what the text itself must contain.
First, make a folder (in the main Forms project) called matching. We will put three classes in there: Constants.cs, BasicHandler.cs, and TextHandler.cs. First, let’s cover Constants.cs:
public class Constants
{
public enum Protocols
{
RAW_TEXT = 0,
SEND_PARTIAL_STAT = 49,
GET_NEW_INVENTORY_ITEM = 21,
}
public enum Colors
{
c_green1 = 3,
c_red1 = 0,
}
public enum Channels
{
CHAT_SERVER = 3,
}
}
This class simply contains some useful constants used to identify channels, colors, and protocols. We’ve included only the bare minimum; please add any constants as your message matching needs expand.
Now, BasicHanlder.cs:
class BasicHandler : Eternal_Lands_Connector.MessageHandler
{
public delegate void OnMatchDel(byte[] data);
private int protocol;
private OnMatchDel OnMatch;
public BasicHandler(int protocolMatch, OnMatchDel OnMatch)
{
this.protocol = protocolMatch;
this.OnMatch = OnMatch;
}
public override int getProtocol()
{
return protocol;
}
public override bool performMatch(int protocol, byte[] data, Dictionary<string, object> decorations)
{
//Quick initial match
if (protocol != getProtocol())
return false;
//Always match
OnMatch(data);
return true;
}
}
The unfortunate reality is that this class is so simple that it makes understanding it tricky. It’s very simple:
- The constructor takes a protocol value to match, and an OnMatch delegate to perform when that protocol matches. It saves these.
- The getProtocol() method exists so that our MessageMatcher can store this MessageHandler in the correct hash table entry.
- The performMatch() function always returns true, after performing the OnMatch delegate, assuming the protocol byte matches.
Without thinking too much about it, let’s add the code for TextHandler.cs and compare the two:
public class TextMessage
{
public int channel;
public int color;
public String text;
}
public class TextHandler : Eternal_Lands_Connector.MessageHandler
{
private const String KEY_DECORATIONS = “TextHandler_Decorations”;
public delegate void OnMatchDel(TextMessage m);
private Regex msgPattern;
private int msgColor;
private int msgChannel;
private OnMatchDel OnMatch;
//-1 for “all” on color or channel
public TextHandler(String msgPattern, int color, int channel, OnMatchDel OnMatch)
{
this.msgPattern = new Regex(msgPattern);
this.msgColor = color;
this.msgChannel = channel;
this.OnMatch = OnMatch;
}
public override int getProtocol()
{
return (int)Protocols.RAW_TEXT;
}
public override bool performMatch(int protocol, byte[] data, Dictionary<String, Object> decorations)
{
//Quick check; protocol
if (protocol != getProtocol())
return false;
//Get
if (!decorations.ContainsKey(KEY_DECORATIONS))
{
//Construct
TextMessage t = new TextMessage();
t.channel = data[0];
t.color = data[1] – 127;
//Parse Text
int count = data.Length – 2;
t.text = System.Text.ASCIIEncoding.ASCII.GetString(data, 2, count);
//Later, of course, we’ll have to deal with things like embedded color tags, etc.
//Save
decorations[KEY_DECORATIONS] = t;
}
TextMessage message = (TextMessage)decorations[KEY_DECORATIONS];
//Match -Stage 1
if ((msgChannel != -1 && message.channel != msgChannel) || (msgColor != -1 && message.color != msgColor))
return false;
//Does the text match?
if (!msgPattern.IsMatch(message.text))
return false;
//It matches!
OnMatch(message);
return true;
}
}
You’ll notice that I didn’t include the using directives or the enclosing namespace. Whatever Visual Studio provides as defaults will work in this case. You’ll have to add two additional using directives, however. Note the second one; it assigns an alias to an enum within our Constants class. This allows us to simply reference Protocols.RAW_TEXT, instead of the more cumbersome Constants.Protocols.RAW_TEXT.
using System.Text.RegularExpressions;
using Protocols = WindowsFormsApplication1.matching.Constants.Protocols;
We’re now ready to create some message handlers and log their output. Go to your main form’s ConnectToServer method, and add the following lines after your various checks for “Started”
//Make a Message Matcher to handle all our special cases
MessageMatcher match = new MessageMatcher();
match.UpdateConsole = UpdateConsole;
//Add some filters
TextHandler h1 = new TextHandler(“You stopped harvesting.”, (int)Colors.c_red1, (int)Channels.CHAT_SERVER, new TextHandler.OnMatchDel(On_Harvest_Stop));
match.addHandler(h1);
BasicHandler h2 = new BasicHandler((int)Protocols.GET_NEW_INVENTORY_ITEM, new BasicHandler.OnMatchDel(Got_Item));
match.addHandler(h2);
You will also need some using directives:
using WindowsFormsApplication1.matching;
using Protocols = WindowsFormsApplication1.matching.Constants.Protocols;
using Colors = WindowsFormsApplication1.matching.Constants.Colors;
using Channels = WindowsFormsApplication1.matching.Constants.Channels;
…the “WindowsFormsApplication1” should be the name of your project; you should be able to resolve naming conflicts like this on your own. Finally, add your MessageMatcher into the call to startAll:
middleWare.startAll(srv, match, loopbackInfo, txtPathToELFolder.Text, new ShowConnectMsgDel(ShowConnectMsg), new AddToConsoleDel(UpdateConsole), new IncrPacketCountDel(IncrPacketCount), Thread.CurrentThread.ManagedThreadId);
The only thing left to do is to implement On_Harvest_Stop and Got_Item. Here are the methods:
private void On_Harvest_Stop(TextMessage tm)
{
//Show message:
UpdateConsole(“\r\nYou have stopped harvesting.”);
}
private void Got_Item(byte[] data)
{
//Decode our message
int image_id = data[0] | (data[1]<<8);
int quantity = data[2] | (data[3]<<8) | (data[4]<<16) | (data[5]<<32);
int pos = data[6];
uint flags = data[7];
//Show message:
UpdateConsole(“\r\nYou got “ + quantity + ” item(s): “ + image_id);
The On_Harvest_Stop method simply informs the player that he or she has stopped harvesting. At this point, all we have to do is show a window and we’d be at the same point we were with the previous approach. (We’ll spice that up in the next tutorial). The Got_Item method gets the image_id, quantity, position, and flags for the item the user picked up (through harvesting, from a drop bag, whatever) and informs the user of these values. We’ll expand on this in the next section. How do we get this data? Read the Eternal Lands source code –it’s pretty easy to find what you want. For example, a little digging reveals the purpose of the flags byte:
is_resource = binary(0010) //Can be used to manufacture
is_reagent = binary(0001) //Can be used in magic
use_with_inventory = binary(1000) //Item can be used with inventory
is_stackable = binary(0100) //The item is stackable
Moving on, compile and run your code, log in to the main server, and harvest something. Alt+Tab to our C# window and wait for the “You have stopped harvesting” message. Useful, eh? You’ll also see some “Got Item” messages.
Several Item and Harvest Messages
You should take some time to study these messages. First of all, you’ll notice that the quantity of a “got” item is the total number you now have, not the number you just picked up. Second, you’ll see that one of the items we were harvesting (id 35) was harvested at 2x the normal rate. This is because we were at one of the 2x spots for that item.
Step 5: Bonus! Fun With Images
At this point, we’re basically done with phase one of our project. We can hook the server and filter messages by a variety of criteria (without recompiling the connection code). But before we call it quits, there’s one thing that’s bothering me: what exactly is item 35? Unfortunately, there’s no easy way to tell, as the Eternal Lands client simply deals with images and identifiers. But maybe we can at least show a picture of the item we just picked up?
If you dig a bit in the source code, you’ll find that the following code is used for drawing items:
cur_item_img=item_list[pos].image_id%25;
u_start=0.2f*(cur_item_img%5);
u_end=u_start+(float)50/256;
v_start=(1.0f+((float)50/256)/256.0f)-((float)50/256*(cur_item_img/5));
v_end=v_start-(float)50/256;
//get the texture this item belongs to
this_texture=get_items_texture(item_list[pos].image_id/25);
get_and_set_texture_id(this_texture);
glBegin(GL_QUADS);
if(mini)
draw_2d_thing(u_start,v_start,u_end,v_end,mouse_x-16,mouse_y-16,mouse_x+16,mouse_y+16);
else
draw_2d_thing(u_start,v_start,u_end,v_end,mouse_x-25,mouse_y-25,mouse_x+25,mouse_y+25);
glEnd();
Converting this to pseudo-code, we get:
image_id = curr_item_img + (25*item_texture_num)
curr_item_img = u_comp + (5*v_comp)
u_start = 0.2*u_comp
u_end = u_start + 0.195 //Width is roughly in .2 increments
v_start = 1.00076 – 0.195 * v_comp
v_end = v_start – 0.195
A shrewd observer (or one accustomed to OpenGL) will notice that this code assumes Cartesian coordinates, so we’d have to flip the v_* variables.
Before we convert this to C#, browse to the Eternal Lands directory, and look in the “textures” sub-folder. Then, look at any of the “items*.bmp” files. Hmm… these images look mighty familiar…
A 5x5 Grid of Eternal Lands Items
Here’s the code to open one of these 5×5 items files, and read each separate image into a Bitmap object. Add this to your main Form’s ReadELFiles method, right before setting cmbServers.Enabled to true.
//Now load our bitmaps from the textures folder
for (int img_texture_id = 0; true; img_texture_id++)
{
//Load this image
String path = txtPathToELFolder.Text + “\\” + “textures” + “\\” + “items” + (img_texture_id + 1) + “.bmp”;
if (!File.Exists(path))
break;
Bitmap patch = new Bitmap(path);
//Loop through all possible images
for (int curr_item_img = 0; curr_item_img < 25; curr_item_img++)
{
//Construct a composite id
int composite_id = curr_item_img + 25 * img_texture_id;
//Get the sub-images’ co-ordinates
float u_start = 0.2F * (curr_item_img % 5);
float u_end = u_start + (float)50 / 256;
float v_start = (1.0f + ((float)50 / 256) / 256.0f) – ((float)50 / 256 * (curr_item_img / 5));
float v_end = v_start – (float)50 / 256;
//Convert to pixels, non-Cartesian
int xStart = (int)(u_start * 256);
int xEnd = (int)(u_end * 256);
int yStart = 256 – (int)(v_start * 256);
int yEnd = 256 – (int)(v_end * 256);
//Draw this into its own bitmap
Bitmap pic = new Bitmap(xEnd – xStart, yEnd – yStart);
Graphics g = Graphics.FromImage(pic);
g.DrawImage(patch, new Rectangle(0, 0, xEnd – xStart, yEnd – yStart), new Rectangle(xStart, yStart, xEnd – xStart, yEnd – yStart), GraphicsUnit.Pixel);
itemPics[composite_id] = pic;
}
}
The code is pretty self-explanatory; it simply reads all items*.bmp files until one doesn’t exist. For each of the 25 items, it applies the segmentation algorithm. It does require one more variable, added at the top of the class:
private Dictionary<Int32, Bitmap> itemPics = new Dictionary<int, Bitmap>();
Now, let’s use this! Add the following code to the end of Got_Item, replacing the existing call to UpdateConsole:
UpdateGotImage(image_id);
Now, implement our delegate and method call, as usual:
private delegate void UpdateGotImageDel(int image_id);
private void UpdateGotImage(int image_id)
{
if (justGotItem.InvokeRequired)
justGotItem.Invoke(new UpdateGotImageDel(UpdateGotImage), image_id);
else
{
justGotItem.Image = itemPics[image_id];
}
}
This references a GUI component that doesn’t exist yet. Add a PictureBox to your Form (anywhere it fits) and set its name to justGotItem and its size to 50,50. Sorry for approaching this in a rather roundabout fashion, but I’m sure you were able to keep up. Compile and run your code. Now, pay attention to your C# window every time you get a new item.
Harvesting Roses; Game Proves It's Working
Proof that this actually works is somehow so much more satisfying once images are involved. I’m at a loss to explain it.
The possibilities from here on in are really endless. What about catching Private Messages from players, and funneling them into GTalk through their API? How about updating your Eternal Lands blog with the item you are harvesting or making, in real-time? What about saving the value returned each time you use an Astrology Indicator, and then charting the results to see how your luck fares over time? All of these things are possible, and none (as near as I can tell) violate the terms of use of Eternal Lands. But make sure you post on the EL forum, if you’re not sure. It’s better to get permission rather than get banned.
I regret that I’ve only posted half the code I’ve written at this point. It just takes so long to re-write and re-test everything. I’m a bit tired of all this right now; next time I feel like writing 50 pages of text, though, I’ll finally show you how to soup up your message interface. Consider the following screenshots as teasers. Note the cool “dissolve in” effect, and the “maximize” and “close” buttons (I edited the image; normally they only highlight when you mouse over them.) All of this code works; I just need to document it, and hook up the notifications to real messages (no, I did not get three enrichment stones in a row. 
)
Nice System Tray Icon
Version 2.0 Has a Fancy, Animating Notification Stack
Final Discussion
Eternal Lands is one of my favorite online games, and yet like all cross-platform games, it doesn’t feel tightly integrated into Windows. One benefit of them opening the source to developers is that we can now hack around and improve the code as we see fit, distributing our changes to benefit everyone. In this article, I showed you how to enhance the notifications provided by Eternal Lands, to make harvesting of high-level items less tedious. By using a high-level language (C#) with low-level hooks into Win32 networking, we achieve both power and speed, with the potential to add nice animations and Windows-specific polish as well. Best of luck with your own hacks!
License – Preamble
The current code cannot be used to abuse the terms of use of Eternal Lands, and I’ve chosen a carefully-worded license to prevent further modifications of this code from being used for abuse. The Eternal Lands developers trusted us with their code, and I intend to honor that trust. Please do the same. The following license applies to source code, compiled binaries, and this article, describing how to make the program from scratch. All future source and binary releases (as well as updates to this article) will remain under this license.
License
Copyright (c) 2009 Seth N. Hetu
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The Software may not be modified to abuse the Terms and Conditions of the Eternal Lands game, as expressed in its license (http://www.eternal-lands.com/page/license.txt).
The Software may not be used to inject events to, nor remove messages from, the regular stream of packets sent between the Eternal Lands client and the Eternal Lands server.
All messages sent from the Eternal Lands client to the Eternal Lands server and vice-versa must arrive at their intended destinations without delay and without reordering or modifying them.
The Thread Priority of the event handler used by the Software may not be modified; moreover, any threads created that interact with the Software in any way MUST have a lower Thread Priority than that of the event handler.
If, at any time, an Eternal Lands moderator (as listed at: http://www.eternal-lands.com/page/developers.php?#mods) or Eternal Lands creator (http://www.eternal-lands.com/page/developers.php?#creators) requests that the Software or any specific derivative work be removed, for any reason, this demand must be met, and all source code and compiled code for the Software must be removed from public display. (Aside: We would ask moderators to be reasonable when demanding that the Software be removed.) The Software and its sources may be restored to public display only with the permission of an Eteranl Lands moderator. In the case of a dispute between moderators and creators, the creators are given preference.
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
) Browse to the directory you saved Zcompress to. Make sure you have also saved a clean Zelda3.smc ROM file there as well. Now, type:
