Performing with Apache JMeter, the Ruby Way

Things were as usual

Although I am not a huge fan of JMeter but somehow every time when there is a need for performance testing in my projects, the tool that I end up with, is JMeter. JMeter, as a performance testing tool, to me is like a diamond wrapped in shit. If you dare to clean up the outer mess, you get a fantastic tool to work with. And that’s what I was doing in my projects. I did not have to do an extensive performance testing yet, but till now JMeter worked pretty good. If you have used JMeter previously, you possibly know that it has a UI, which looks like built in 1800s by a very lazy developer. But I could live with that. The world is not always the way you want it to be! The pain point of the tool is the output the tool gives. Its script is with an extension “.jmx” which is nothing but an XML. Now if you are working with JMeter in a headless environment, you have to deal with this XML. To make minimal modification to the script, you have to parse the XML and perform the necessary changes. Or, you have to download the script to a “Head-Full” environment, open JMeter UI, make the changes and re-upload the “.jmx” file in the “Headless” environment. Given the minimal amount of performance testing I had to do, I was fine even doing that too, until…

The Problem

In my current project, there was a need to provide some values to JMeter before starting the tests. For example, creating a job in Jenkins and providing parameters like, “Number of Threads”, “Ramp Up”, “Iteration” etc dynamically. Why? well, thats a separate story and may be sometimes later, but given that JMeter supports “jmeter” command to run your tests from command line, this seems to be a very reasonable request, isn’t it? But guess what, JMeter does not support this. When I did a “jmeter --help” from my terminal, I found no options to do this. Then further research in different discussions in forums, I found out thats true.

The Research

The research for the solution started with searching for a way to update an XML file dynamically. I thought this will be the only solution where I parametrize the “number of thread” tag in “.jmx” file and pass this dynamically through Jenkins. But I was frustrated after spending time in googling to find if that’s actually possible. I was trying to search over and over again for someone who had a similar solution and blogged about his experience. (Later I thought, why would someone think of a solution like this? That’s Stupid! Updating XML dynamically? Come On!). Atlast I found about this gem called “ruby-jmeter” which actually lets you write JMeter script as RSpec type tests. I was amazed to see that it’s even possible. Although, the discussion that I was reading was not exactly about solving my problem, but I thought I must give it a try, looking at the neatness of the “Ruby-JMeter” tests.

The Solution

That gem is a “Gem”! In an hour, I had my Ruby-JMeter scripts ready and pushed to github and I now have a parametrized JMeter script so that I can pass all the values for running the scripts right from Jenkins. Now the scripts look pretty neat and easy to maintain, because I don’t have to go edit the nesty XML if I have to make some changes to the JMeter scripts. Its all Ruby coding now. This is how my new so called JMeter script looked like:

[Note: This is not actual code, I created a sample for the blog]

I created a performance helper, to extract the required number of values passed to the ruby script from command line through ARGS.

Wait, it can do more!

Now that I am done with my objective, it’s time to make the code look a bit beautiful. So how the jmeter ruby scripts work is, when you run the ruby file containing the script, it parses the wrappers and creates a “.jmx” file and runs it using ‘jmeter” command. Now I do not need that “.jmx” file as a part of my code base because the only way I am interacting with JMeter is through the ruby scripts using those beautiful wrappers. Also, the script create a “.jtl” file as a report of execution, but If I do not clear it before every run, the results get appended to the previous one. So, I created a shell script to handle these scenarios. Also, it looks a bit neater when you run the shell script from Jenkins instead of a confusing command. So, came up with this:

The design

Last but not the least, it’s time to organize all these different files into different folders so that it looks a bit organized, or on a fancy term, creating a Framework. So, after a bit of brainstorming with folder names, and which file should go where, this how the folder structure or framework looked like:

A small catch

There is a small catch though. Remember, I said the run creates a “.jtl”/”.xml” report? Well, I would have loved to see a parser which parses this to a nice HTML report. Currently, to see the report, you have to open JMeter, create a listener and open the file there. Which is not much of a pain compared to what I was dealing with, but yes, probably a nice to have feature.

A little learning to help solve this

Well, although I said this is a catch of the gem, this now gives me an opportunity to learn new stuffs in HTML. What I am currently working on is to create a fancy HTML report for this. The work is in progress and going on well. To give a brief overview, I created a json parser to parse the report to a json. Now I am working on creating an HTML using the Handlebar templates and populate the DHTML with the json values to make this a complete package.

Conclusion

I am very thankful to the guys at Flood.io to come up with such a gem which gave me my first performance testing framework. You can do everything with this gem, that you can do with JMeter, plus it saves you from the XMLs. You can find the project here in github, with a very good documentation. Let me know how you feel about it and ping me anytime if you want to pair on Handlebar!