[]Charts of performance/errors (bar chart)
2 Weeks ago I was getting for ready for TechEd Demojam and since my demo was going to involve the audience actually using the a website being served up by the XS engine, I started having some concerns about how much load it could handle. I spent some time digging through the documentation as well as reaching out to guys like Thomas Jung for any advice I could get. As always, it was an interesting experiment and thought it would be great to share some of the experiences I had, and hopefully you can share some ideas or feedback from your point of view.
So, just add some context – the website the audience was going to interact with at Demojam was extremely simple, a single page, 2 buttons, when a user clicked on one of the buttons, I was making a call to a XSJS file, doing a table insert and returning a confirmation, (and in the demo some counts). Pretty simple and straight forward.
As an estimate, I was anticipating around a thousand users would actually perform the task. Generally Demojams have anywhere between 1,000 and 2,000 attendees and used this as my guide. I figured that if 1/3 of the audience actually did what I asked, that would probably be quite realistic. (My estimates were close as in the end, as the audience made +- 700 “purchases”)
So onto the pages and technology behind the scenes ….
SAPUI5
I originally had the page styled using SAP UI5 to correlate with my Demo UI, but due to the size of the JS Libraries, I quickly ripped it out to keep page loading speed to a minimum.
Next, I added just a JQuery library in the place of UI5 to keep the convenience of the AJAX libraries, but it too was a little larger than I was hoping for (Around 90Kb). So I decided to stick with a traditional/Old school XML RPC call in Javascript, this mean’t the page being loaded was 52.1KB (including images, scripts, CSS and the HTML).
I did use Twitter Bootstrap to help with the responsive/multi-device aspects and trimmed the CSS down a bunch using a tool for Firefox/Firebug called “CSS Usage”, a very helpful tool which essentially checks all your CSS sheets and tell you which styles are/are not being used.
The following lines of code were in my XSJS file, pretty straight forward since all I was doing was a simple increment/insert for tracking purposes.
$.response.contentType = “text/plain”;
var intChoice = $.request.parameters.get(‘choice’);
var strDashboardUser = “lilabs::storeuser”;
var SQL = ”;
var strHTML = ”;
if (intChoice){
executeQuery(intChoice);
}
function executeQuery(intChoice){
var conn = $.db.getConnection(“lilabs::storeuser”);
SQL = ‘INSERT INTO store.m2_pill (choice) VALUES (?)’;
var pstmt = conn.prepareStatement(SQL);
pstmt.setInteger(1, parseInt(intChoice));
var rs = pstmt.execute();
conn.commit();
}
$.response.setBody(strHTML);
JMeter and Load testing
Once I had my files at a reasonable size for a variety of mobile devices, I started looking for services to load test my site. I wanted a free option and although many sites offered this, it was generally for very low user numbers or test counts, or in some cases, not compatible with XS Engine. Not compatible you say … ? Since the load testers essentially can cause a DOS attack they often require you to place a token somewhere in the root domain of your site. Since you technically cannot place a file in the root of a XS Engine/Native HANA app, this leads to some issues in fulfilling this prerequisite.
In the end I settled with JMeter whish is not a service, but easy enough to configure on your own, here is an excerpt from the wiki page to give you a little overview:
Apache JMeter is an Apache project that can be used as a load testing tool for analyzing and measuring the performance of a variety of services, with a focus on web applications. JMeter can be used as a unit test tool for JDBC database connections, FTP, LDAP, Webservices, JMS, HTTP, generic TCP connections and OS Native processes.
I setup a “m1.Large” Windows Instance in AWS and subsequently installed JMeter, the install is a breeze with one minor requirement being JRE. Once I had JMeter installed, I created a couple of different test scenarios for my XS Site. The first was just the initial “Index” page load, and the second an actual action against the DB simulating someone clicking on one of the selections. These are fairly simple to setup in JMeter:
XS Engine Tuning
I originally tested with the “Out of the box” settings which are defined for a “Medium” instance, some of the more important items I was focused on included:
XS Engine config:
maxsessions
maxthreads
Web Dispatcher Config:
icm/max_conn
icm/max_sockets
icm/max_threads
wdisp/HTTP/max_pooled_con (or HTTPS)
Here are a couple of the result sets which I used to gauge my thresholds/requirements:
Users/Threads: 1,000
Ramp Up Period: 30 secs
“Medium” Setup
Configs: As above
Server: AWS m2.2xlarge
This setup seems to work pretty well for around concurrent 1,000 users and subsequently starts to throw a couple of exceptions and ultimately causes XS Engine to choke at around 1,500 users.
“Large” Setup
Configs: As above
Server: AWS m2.4xLarge
Users: 1,000
Ramp Up Period: 30 Secs
As above, pretty close to identical results, not surprising, however the larger server (More RAM and CPU) handles around 4,000 connections reasonably well (with +- 2% error rate):
Users, 4,000 (db.xsjs)
After cranking up the connections to 10,000 …. Well, thats embarrassing 😉
Conclusion
While by no means is this scientific or conclusive, its interesting to see just how much of a load XS Engine can handle. In the end I am extremely confident that an actual HANA appliance would perform way better. The virtual limitations of the CPU is a major factor and also only having 30/60GB RAM respectively on each machine is probably not much help either. I am also pretty impressed, as I creep towards making a web app using XS Engine as a Web/App server, these numbers make me a little bit more confident in its ability to support this scenario.
