Load Testing Metric2 and SAP HANA

[]Charts of performance/errors (bar chart)

2 Weeks ago I was getting for ready for TechEd Demojam and since my demo was going to involve the audience actually using the a website being served up by the XS engine, I started having some concerns about how much load it could handle. I spent some time digging through the documentation as well as reaching out to guys like Thomas Jung for any advice I could get. As always, it was an interesting experiment and thought it would be great to share some of the experiences I had, and hopefully you can share some ideas or feedback from your point of view. 

So, just add some context – the website the audience was going to interact with at Demojam was extremely simple, a single page, 2 buttons, when a user clicked on one of the buttons, I was making a call to a XSJS file, doing a table insert and returning a confirmation, (and in the demo some counts). Pretty simple and straight forward. read more

Open Source Project: HANA Talk – A Simple HANA XS helper

HANA Talk is a small Javascript class which help facilitate the communication between your front end html/js files and HANA database when using SAP 
HANA XS Engine. This is intended to help people who are just starting out development on XS Engine and would like a easy place to get  their feet wet without having to go through too much trial and error. This is also a great starting point for people involved in events like InnoJam, Hackathons, etc. when POC’s and demos need to developed quickly, but not necessarily perfectly *cough* or securely *cough* 🙂 

By providing this tutorial and the HANA Talk download, my hope is that it will encourage more people to consider using XS Engine as a app platform to drive their front end web apps and subsequently, encourage the use and innovation around HANA in general. Anne Hardy had a comment in her blog post regarding the Developer Advisory Board which was along the lines of “Developers want to get it in 5mins max; they want to build real stuff in less than an hour”.

I frequently get discouraged by the amount of learning and effort always needed to learn new technologies and products which companies “get into bed with”. Nearly every one of the apps I develop start out in a POC type phase, where my imagination and reality clash and inevitably produce the equivalent of a 3yr old’s self portrait, often resembling “Unconventional”. While going through these exercises I often wish it was simple to get somethings done, and understanding the cost and willing to accept the trade offs. This is why I developed HANA Talk – to make those unconventional portraits, easier to deliver 🙂 

OK – so what does it do?

By adding a HANA Talk js and xsjs file to your project, you can simply write SQL statements in your HTML file and have the results returned synchronously.

e.g.  Index.html

<script type="text/javascript">
var resp = hana.executeScalar('SELECT 1 FROM DUMMY');
console.log(resp); //Outputs 1
</script>

Super simple and easy. See below for further details.

Prerequisites

1. Download/fork these 2 files – client.js & server.xsjs from Github and add them to your package. The filenames/structure can be changed if you are feeling adventurous. In your HTML file, you will need to reference client.js, this is as simple as adding a tag to you header. For reference, if you are not using SAP UI5 or jQuery – you will need to add this to your HTML header as well.

A Basic Example

2. In your javascript code, instantiate a new HanaTalk object. We will use this to “pass” our SQL commands to our HANA DB.

var hana = new HanaTalk('SYS'); //The 'SYS' reference is in relation to the Schema. It can be specified here or within your TSQL Statement

3. Call your HanaTalk object with the operation type and SQL you would like execute (see below for additional operations).

var result = hana.executeRecordSet("SELECT 1 FROM DUMMY");

4. We can populate that response into our html (DOM)

document.getElementById("SomeElementID").innerHTML = result;

A few more examples

a.) Insert/Update/Delete a record – use .executeChange, this will execute your code and respond with the records which have been updated

document.getElementById("resp4").innerHTML = /*hana.executeChange("UPDATE/INSERT/DELETE .... ") + */ ' Record Changed';

b.) Return a Table – using .executeRecordSet will return a html formatted table, displaying the select’s record set

document.getElementById("resp2").innerHTML = hana.executeRecordSet("SELECT TOP 5 * FROM M_HOST_INFORMATION");

c.) Return a Object – .executeRecordSetObj allows us to loop through records, and have quite a lot of control of the display of each record and its column name.

document.getElementById("resp3").innerHTML = hana.executeRecordSetObj("SELECT TOP 5 * FROM M_HOST_INFORMATION"); read more

SAP UI5 – A Photoshop template of GUI elements

32 43 19,722 

Mockup.png

For me (and I am sure many others!), UI5 has become a really useful toolkit for easily developing web apps and POC’s with little effort around the SAP Ecosystem. The OData support, rich controls and business inspired examples lend itself well to the mantra of “Don’t reinvent the wheel”. At a recent SAP Inside Track event in NY, I made reference to the fact, that while developing a mobile app, I, or a graphic designer, can spend close to 40% of the total design & development phase/hours on the UI. Building “consumer grade” enterprise apps is a different way of thinking for app developers. I am sure many of you, like me, started out with a redefined canvas of boring, grey .NET controls or a “Enjoy” SAP CXTAB_CONTROL. But times are changing and we should all look to the creative and vibrant consumer world of both web and mobile apps to reconsider our methods and processes for developing great apps. read more

A peek inside xSync and the HANA XS Engine

/wp-content/uploads/2013/06/icon_128x128_231789.png

On saturday I published a blog about a small app I wrote called xSync – basically a XS Engine app for Mac developers where you can sync a local development folder with your HANA repository. This is for rapid development and to encourage the “bring your own IDE” approach to application development on HANA. Here is a look behind the scenes on how the app works and some of the challenges of the project.

Image.png

As mentioned in my previous blog – I started using the IDE Lightweight editor after doing the upgrade of my AWS HANA box last weekend. I enjoyed the experience but after working with it for nearly a full day was wanting a little more. Syntax highlighting, easy commenting, easy indentation, CSS autocomplete and hints, etc. etc. so I started doing some peaking around the editor itself and came to find the editor is something called ACE, a pretty nice little open source IDE (written in JS). This got me thinking … maybe I could insert text directly into the Lightweight IDE browser text box, and submit the form as a save …. hmmm …. not a terrible idea …. just need to scrape the page, find the elements and submit the form via some injected JS. Pretty simple …  I did some digging and found the HTML objects I needed by using Firebug when a lightbulb went off … instead of populating the form via a HTML page, why not rather check the HTTP methods it is calling when doing the actual save, since there must be some integration with HANA directly … which is when I came across the mother load … a small file called reposervice.xsjs 🙂 It seemed that every time I was saving or modifying my objects through the IDE, it was calling this file. After checking out the parameters it was `, it was very clear that the methods and text were easy to simulate. I fired up REST Client and within a couple minutes the concept was POC’ed. Pass your file contents as your body with a path param and a POST and you were off to the races 🙂 

Screen Shot 2013-06-09 at 4.39.png

Using Firefox Rest Client to monitor system calls showed each save, create, delete operation was using a small file called reposervice.xsjs, which references the libraries needed for the repository modifications. read more

xSync – Making HANA XS development easier

Update: March 7th 2014

– Added support for SAP HANA SPS07 
– Ability to ignore local deletes, which would leave the files on the server 
– Shortcut link opens the Web Editor, and another to open IDE in your browser 

– Stability and speed improvements 
– Corrected multiple minor bugs  

————————————————————————————————————————————————

A couple of weeks ago, I started playing with HANA development after going through a couple of the opensapcourses. One of my biggest surprises was that HANA Studio was windows only. As a Web/Mobile/App developer spending majority of my time on the Mac platform, having to get my Dell out was a walk down memory lane, and caused me to not spend as much time working on the projects as I would have liked. Since Rev 52 of HANA included a XS Engine Lite IDE, a useful web based IDE for your XSJS/SAPUI5 development. I was been looking forward to getting it installed on my AWS machine and giving it a test drive. Last weekend Juergen Schmerderposted a updated really easy to follow and simple guide of going through the process and within a hour, I was up and running on the newest available rev. read more

Simple Sample: Writing an entry to SAP Netweaver Gateway using JSON

Skill Level: Beginner

Time Needed: 20 minutes

I recently assisted someone on the forums when they were having trouble creating records in their SAP ECC instance while using Gateway. I thought I would share some very simple code showing what it takes to write an entry into a service via JSON from iOS (Objective C). The reason the title of the document is “Simple Sample” – there are no frills. Whenever I develop a new app, I start small, get the basic functionality working, and proof out the process before getting too creative.

*** If you do not have your own gateway server, you might be able to connect this to the SAP Gateway System which can be found hereCreate an account and utilize this service *** DISCLAIMER: Untested as the signup form does not work 🙁  read more

Quick View for SAP Notes

In my ever growing quest to provide quick and easy access to SAP Notes (which Dagfinn Parnas started a couple of years ago with the Android Note Viewer), I recently finished creating a windows version of a small app to make opening notes just a little easier.

When deciding if I should get this done, it was purely out of personal need. Having spent the last few months building up a couple of Gateway test, stage and Prod systems, the ability to open a SAP note quickly, always seemed to elude me. Open service.sap.com/support -> click on Help & Support -> Type the note and go. Albeit the app developed still has some user interaction (until Google Glass arrives?) … it does make it a little simpler. I also found a small cmd script online which looked helpful. read more

Practical Big Data Use Cases

Background

Source:

https://www.kaggle.com/wiki/DataScienceUseCases

For each type of analysis think about:

  • What problem does it solve and for who?
  • How is it being solved today?
  • What are the data inputs and where do they come from?
  • What are the outputs and how are they consumed- (online algo, static report, etc)
  • Is this a revenue leakage (“saves us money”) or a revenue growth (“makes us money”) problem?

Use Cases By Function

Marketing

  • Predicting Lifetime Value (LTV)
    • what for: if you can predict the characteristics of high LTV customers, this supports customer segmentation, identifies upsell opportunties and supports other marketing initiatives
    • usage: can be both an online algorithm and a static report showing the characteristics of high LTV customers
    Wallet share estimation
    • working out the proportion of a customer’s spend in a category accrues to a company allows that company to identify upsell and cross-sell opportunities
    • usage: can be both an online algorithm and a static report showing the characteristics of low wallet share customers
    • Churn
    • working out the characteristics of churners allows a company to product adjustments and an online algorithm allows them to reach out to churners
    • usage: can be both an online algorithm and a statistic report showing the characteristics of likely churners
    • Customer segmentation
    • If you can understand qualitatively different customer groups, then we can give them different treatments (perhaps even by different groups in the company). Answers questions like: what makes people buy, stop buying etc
    • usage: static report
    • Product mix
    • What mix of products offers the lowest churn? eg. Giving a combined policy discount for home + auto = low churn
    • usage: online algorithm and static report
    • Cross selling/Recommendation algorithms/
    • Given a customer’s past browsing history, purchase history and other characteristics, what are they likely to want to purchase in the future?
    • usage: online algorithm
    • Up selling
    • Given a customer’s characteristics, what is the likelihood that they’ll upgrade in the future?
    • usage: online algorithm and static report
    • Channel optimization
    • what is the optimal way to reach a customer with cetain characteristics?
    • usage: online algorithm and static report

    Discount targeting – What is the probability of inducing the desired behavior with a discount – usage: online algorithm and static report

    • Reactivation likelihood
      • What is the reactivation likelihood for a given customer
      • usage: online algorithm and static report
      Adwords optimization and ad buying
      • calculating the right price for different keywords/ad slots

      Sales

      • Lead prioritization
        • What is a given lead’s likelihood of closing
        • revenue impact: supports growth
        • usage: online algorithm and static report
        Demand forecasting

        Logistics

        • Demand forecasting
          • How many of what thing do you need and where will we need them? (Enables lean inventory and prevents out of stock situations.)
          • revenue impact: supports growth and militates against revenue leakage
          • usage: online algorithm and static report

          Risk

          • Credit risk
          • Treasury or currency risk
            • How much capital do we need on hand to meet these requirements?
            Fraud detection
            • predicting whether or not a transaction should be blocked because it involves some kind of fraud (eg credit card fraud)
            Accounts Payable Recovery
            • Predicting the probably a liability can be recovered given the characteristics of the borrower and the loan
            Anti-money laundering
            • Using machine learning and fuzzy matching to detect transactions that contradict AML legislation (such as the OFAC list)

            Customer support

            • Call centers
              • Call routing (ie determining wait times) based on caller id history, time of day, call volumes, products owned, churn risk, LTV, etc.
              Call center message optimization
              • Putting the right data on the operator’s screen
              Call center volume forecasting
              • predicting call volume for the purposes of staff rostering

              Human Resources

              • Resume screening
                • scores resumes based on the outcomes of past job interviews and hires
                Employee churn
                • predicts which employees are most likely to leave
                Training recommendation
                • recommends specific training based of performance review data
                Talent management
                • looking at objective measures of employee success

                Use Cases By Vertical

                Healthcare

                • Claims review prioritization
                  • payers picking which claims should be reviewed by manual auditors
                  Medicare/medicaid fraud
                  • Tackled at the claims processors, EDS is the biggest & uses proprietary tech
                  Medical resources allocation
                  • Hospital operations management
                  • Optimize/predict operating theatre & bed occupancy based on initial patient visits
                  Alerting and diagnostics from real-time patient data
                  • Embedded devices (productized algos)
                  • Exogenous data from devices to create diagnostic reports for doctors
                  Prescription compliance
                  • Predicting who won’t comply with their prescriptions
                  Physician attrition
                  • Hospitals want to retain Drs who have admitting privileges in multiple hospitals
                  Survival analysis
                  • Analyse survival statistics for different patient attributes (age, blood type, gender, etc) and treatments
                  Medication (dosage) effectiveness
                  • Analyse effects of admitting different types and dosage of medication for a disease
                  Readmission risk
                  • Predict risk of re-admittance based on patient attributes, medical history, diagnose & treatment

                  Consumer Financial

                  • Credit card fraud
                    • Banks need to prevent, and vendors need to prevent

                    Retail (FMCG – Fast-moving consumer goods)

                    • Pricing
                      • Optimize per time period, per item, per store
                      • Was dominated by Retek, but got purchased by Oracle in 2005. Now Oracle Retail.
                      • JDA is also a player (supply chain software)
                      Location of new stores
                      • Pioneerd by Tesco
                      • Dominated by Buxton
                      Product layout in stores
                      • This is called “plan-o-gramming”
                      Merchandizing
                      • when to start stocking & discontinuing product lines
                      Inventory Management (how many units)
                      • In particular, perishable goods
                      Shrinkage analytics
                      • Theft analytics/prevention (http://www.internetretailer.com/2004/12/17/retailers-cutting-inventory-shrink-with-spss-predictive-analytic)
                      Warranty Analytics
                      • Rates of failure for different components And what are the drivers or parts?
                      • What types of customers buying what types of products are likely to actually redeem a warranty?
                      Market Basket Analysis Cannibalization Analysis Next Best Offer Analysis In store traffic patterns (fairly virgin territory)

                      Insurance

                      • Claims prediction
                        • Might have telemetry data
                        Claims handling (accept/deny/audit), managing repairer network (auto body, doctors) Price sensitivity Investments Agent & branch performance DM, product mix

                        Construction

                        • Contractor performance
                          • Identifying contractors who are regularly involved in poor performing products
                          Design issue prediction
                          • Predicting that a construction project is likely to have issues as early as possible

                          Life Sciences

                          • Identifying biomarkers for boxed warnings on marketed products
                          • Drug/chemical discovery & analysis
                          • Crunching study results
                          • Identifying negative responses (monitor social networks for early problems with drugs)
                          • Diagnostic test development
                            • Hardware devices
                            • Software
                            Diagnostic targeting (CRM) Predicting drug demand in different geographies for different products Predicting prescription adherence with different approaches to reminding patients Putative safety signals Social media marketing on competitors, patient perceptions, KOL feedback Image analysis or GCMS analysis in a high throughput manner Analysis of clinical outcomes to adapt clinical trial design COGS optimization Leveraging molecule database with metabolic stability data to elucidate new stable structures

                            Hospitality/Service

                            • Inventory management/dynamic pricing
                            • Promos/upgrades/offers
                            • Table management & reservations
                            • Workforce management (also applies to lots of verticals)

                            Electrical grid distribution

                            • Keep AC frequency as constant as possible
                            • Seems like a very “online” algorithm

                            Manufacturing

                            • Sensor data to look at failures
                            • Quality management
                              • Identifying out-of-bounds manufacturing Visual inspection/computer vision
                              • Optimal run speeds
                              Demand forecasting/inventory management Warranty/pricing

                              Travel

                              • Aircraft scheduling
                              • Seat mgmt, gate mgmt
                              • Air crew scheduling
                              • Dynamic pricing
                              • Customer complain resolution (give points in exchange)
                              • Call center stuff
                              • Maintenance optimization
                              • Tourism forecasting

                              Agriculture read more

SAP Netweaver Gateway T-Code Summary

Here is a list of useful SAP Netweaver Gateway TCodes, please add a comment for any I am missing and I will gladly update. The intent of this post is to summarize the relevant TCodes for posterity and to be used as a quick lookup.

Gateway Runtime

TitleT CodeDescriptionSP Required
Gateway Client/IWFND/GW_CLIENTSAP NetWeaver Gateway is a powerful tool that enhances the existing Error Log. Also allows you to simulate service execution. (Similar to tools like RESTClient for Firefox)SP04 +
Error Log/IWFND/ERROR_LOGThe Error Log for SAP NetWeaver Gateway hub systems is a helpful addition to the existing Application Log Viewerand provides detailed context information about errors that have occurred at runtime.
Application Log/IWFND/APPS_LOG (Gateway Hub System)Log and Trace Filesa specific viewer for application log entries created by SAP NetWeaver Gateway processes.
Application Log/IWBEP/VIEW_LOG (Business Suite Backend)Log and Trace Filesa specific viewer for application log entries created by SAP NetWeaver Gateway processes.
Application Log Cleanup/IWFND/CLEANUP_APPSLOGOver time the application log amasses a lot of entries, which periodically need to be cleared. As default the application log should only store one entry per request. In addition, there is no need to store these entries for more than a day or a few days. Application log cleanup is a standard requirement in productive systems and helps to improve the performance of the systems in your system landscape.
Clear Cache/IWFND/CACHE_CLEANUPCleanup Gateway Metadata Cache
Disable Cache/IWFND/MED_ACTIVATEDisable Gateway Metadata Cache
Gateway Node ActivationSICFIncludes various functions for monitoring, analysis, and troubleshooting in the ICF environment. You can also make use of the central Application Server analysis functions to identify issues such as performance problems in ICF applications.
Alert MonitoringRZ20SAP NetWeaver Gateway is monitored automatically within the Computing Center Management System (CCMS) in SAP CCMS Monitors for Optional Componentsfor Information Worker.In the monitoring process, information is collected about SAP NetWeaver Gateway. Here you can see which applications are in use, and how often they are called up. The information is updated hourly and relates to today’s date, since on the first collection after midnight all values are initialized once they have been read out.
Performance Trace/IWFND/TRACESThe Support Utilities offering for SAP NetWeaver Gateway has been extended to include a Performance Trace tool. This tool enables developers, administrators, support consultants, and end users to monitor system performance at service call level. You can trace the performance of both the SAP Business Suite backend system and the SAP NetWeaver Gateway hub system.

Gateway Design

TitleT CodeDescriptionSP Required
Gateway Service BuilderSEGWA completely new design-time transaction, which provides developers with an easy-to-use set of tools for creating services. It has been conceived for the code-based OData Channel and supports developers throughout the entire development life cycle of a service.SP04+
Model Composition for Integration/IWFND/BD_MGW_MDL_COMPOSITIONModel composition is provided for complex scenarios for the integration topics, such as SAP Business Warehouse, GenIL and SPI. It allows mashing up multiple services within IW-BEP.
Service Validation/IWFND/SRV_VALIDATESAP NetWeaver Gateway Service Validation Tool validates the runtime for all the SAP NetWeaver Gateway services to ensure that all the services work fine. Service Validation tool runs without user intervention and performs the runtime validation on a single service document.
Register/Test Backend SystemSM59Your backend system has to be registered as an SAP system alias in the SAP NetWeaver Gateway Implementation Guide (IMG) on your SAP NetWeaver Gateway system. In addition, an RFC destination has to be defined in transaction SM59.
Create a ServiceSE80Design time ABAP Workbench
Register a Service/IWFND/REG_SERVICE Replaced with SEGW
Maintain a Service/IWFND/MAINT_SERVICEReplaced with SEGW
Activate a Service/IWFND/IWF_ACTIVATEActivate or Deactivate a service
MOC/IWFND/MOC_PARALLELParallelize Multi-Origin Composition
WebGUI Workflow/IWFND/WF_WEBGUIWebGUI For Executing Workflow Item
Configure Message based Authentication/IWFND/WSS_SETUPConfigure Web Service Message-Based Authentication
GenIL Generator/IWGIL/GENIL_SRV_GEN Gateway Service Generator for GenIL