Notification Actions Engine

From E-COMPASS_Info_Guide
Jump to navigation Jump to search

Physical Hardware Characteristics

Model: Cisco UCS B200 M4 und M2 / Cisco UCS B230 M2
Processor: 2 socket CPUs with 12, 20 or 24 cores
RAM: 256 GB
Hard Drive Space: SAN Storage mirrored HP 3par System 7400c with 100 TB each
Network Connection: between VMs: 10 Gbit, SAN Storage connection: 8 Gbit FibreChannel at minimum, outward: 10 Gbit Ethernet
Hypervisor used: VMware ESX 6 with vSphere Center 6 in the Cluster
Physical Load Balancing: none

Virtual Machine Hardware Specifications and Operating System

Guest Operating System: Microsoft Windows 8 (64bit)
Processor: 4
RAM: 3 GB GB
Hard Drive Space VM: 44 GB
Network Connection: 10 Gbit/s
Minimum required Network Connection: no info available

Service Environment and Set-up on VM

Required Software
Software Download
MySQL Server 5.6 (Community Edition) dev.mysql.com/downloads/mysql/
PIWIK – Open Analytics Platform piwik.org
Java 6.X java.com

Software Licenses

Please indicate if a commercial provider would need to buy commercial licenses of a certain software used for operating the service and – if so – what cost this may produce approximately

OS Environment Variables

JAVA_HOME: path to Java installation e.g. C:\Program Files (x86)\Java\jdk1.8.0_51

Path: should contain path to Java bin folder e.g. C:\Program Files (x86)\Java\jdk1.8.0_51\bin

Installation

Java: Latest Java JDK (not JRE) has to be installed. File and instructions can be found on http://www.oracle.com/technetwork/java/javase/downloads/index.html JAVA_HOME and Path environment variables must be set.
MySQL Community Edition: File and instructions can be found on http://dev.mysql.com/downloads/
Java Jetty Server and Application: Copy .war- and .jar-file (e.g. ProcessEngineWeb-0.0.1-SNAPSHOT.war and ProcessEngineLauncher-0.0.1-SNAPSHOT.jar) in any folder and start the application by clicking on the .jar-file.

Preparation

Before starting the application, a valid properties file has to be provided. An example of such a properties file is attached to this wiki. File:ecompass.txt This property file has to be stored on the virtual machine. To provide the path to this property file to the Java application, you can start the application with the following command: java -jar -DARGS_CONFIG_CEP="C:\ecompass.properties" assuming that the properties file is saved under C with the file name ecompass.properties.

To create the database schema the following file can be used: File:createtables.txt

Service Configuration

The java application communicates via REST services with other modules. Endpoints can be defined in the central properties file.

Configuration script

The configuration script can be put in any directory on the virtual machine. To provide the correct path to the Java application use the following command: java -jar -DARGS_CONFIG_CEP="C:\ecompass.properties" assuming that the properties file is saved under drive C with the file name ecompass.properties.

README / User Manual

The features are described by the embedded help feature within the graphical user interface.

Configuration steps

All main configuration parameters can be found in the properties file. This file contains the base URL under which REST services are exposed (parameter cep.rest.baseUrl) as well as parameters to define endpoints of remote services, which are used by the application.

Properties File

Category Property Description Example
MySQL database jdbc.url database connection URL of the MySQL database for jdbc jdbc:mysql://localhost:3306/cep_core?useUnicode=yes&characterEncoding=utf8
MySQL database jdbc.username database connection URL of the MySQL database for jdbc root
MySQL database jdbc.password database connection URL of the MySQL database for jdbc SeCretPassWord
Piwik piwik.serverUrl URL of the Piwik installation for registration of new users and sites http://example.org/piwik/
Piwik piwik.token Authenticaton token for the Piwik server. The Piwik user of this token, must have admin rights. 1234567890abcdef1234567890abcdef
Mail service mailservice.localhost dns name of the machine with the Notification & Actions Engine e-compass.iao.fraunhofer.de
Mail service mailservice.sender MailService sender name and e-mail adress. SME E-Compass MailService <ecompass@iao.fraunhofer.de>
Mail service mailservice.smtp_host hostname of the smtp server mailhost.iao.fraunhofer.de
Mail service mailservice.smtp_port port of the smtp server 25
Mail service mailservice.smtp_username username for the smtp server - only if needed
Mail service mailservice.smtp_password password for the smtp server - only if needed
ReST endpoints cep.rest.baseUrl URL of the cep service of the Notification & Actions Engine https://e-compass.iao.fraunhofer.de/processengine/
ReST endpoints datacollector.address URL of the Data Collector & Consolidator (Module 2) http://ecompass.khaos.uma.es/ecompass/api
ReST endpoints pricemonitor.address URL of the price monitor of the Competitor's Data Collector http://pricemonitor.iao.fraunhofer.de

Operation

Service startup procedure

To start the Java application it is sufficient to start the .jar file which itself starts a Jetty server and initializes the application as long as a valid properties file is provided.

Restarting the service

To restart the application, the running Java application has to be stopped. This can be done by the following commands in the command line e.g. 

taskkill /f /im jqs.exe
taskkill /f /im javaw.exe
taskkill /f /im java.exe

After this the application can be restarted again by starting the .jar file.

Service Logs

Logs are written to the subdirectory .\logs of the application. E.g. C:\processengine\logs if the engine is started from C:\processengine. There is a new log file for each day. The log file name format is processengine.log.YYYY-MM-DD, e.g. processengine.log.2016-01-25. The log file for today is just called processengine.log.

Recurring Manual Actions / Maintenance Tasks

When a value of a property parameter has to be changed, the application has to be restarted to activate this new value.

Other

Piwik Plugin

The Notification & Actions Engine needs data about the visitors visiting an E-Shops web page. Therefore the Shop has to use Piwik Analytics with the E-Compass Piwik Plugin installed (Read the E-Compass Piwik Plugin wiki page for more information).

Piwik Accounts

E-shop sites in Piwik which have already been created manually need their belonging Piwik accounts to be created manually later. Newly created e-shops can be created together with their corresponding accounts through following RESTful POST-Request:
http://e-compass.iao.fraunhofer.de/processengine/createNewPiwikSite.do

For creating an e-Shop and an account on the Piwik of the Data Collector Consolidator:
(With example values)

Key Description Value
ownerId The id of the e-shop owner in the process engine 1
piwikSiteName Name of the e-shop in Piwik My Shop
piwikMainUrl URL of the e-shop www.my-shop.com
piwikLoginUsername Username of the account which is to be created myshop1
piwikLoginPassword Password of the account which is to be created secretpassword
piwikUserEmail Email of the e-shop owner j.smith@my-shop.com
piwikTimezone Timezone in which the e-shop is operating Europe/Berlin*
piwikCurrency Currency which the e-shop uses EUR**
piwikUserAlias An alias for the e-shop owner MyShop123
userPiwikSiteId Is generated once the e-shop is created left blank
piwikTrackerLocationUrl Uses the URL of the Piwik of the Data Collector Consolidator as default left blank
piwikAuthToken Uses the admin auth_token of the Piwik of the Data Collector Consolidator as default left blank

*,** For a list of all timezones and currencies available in Piwik please refer to following links respectively:
Currencies: http://demo.piwik.org/?module=API&method=SitesManager.getCurrencyList&format=JSON&token_auth=anonymous
Timezones: http://demo.piwik.org/?module=API&method=SitesManager.getTimezonesList&format=Tsv&token_auth=anonymous


Information of E-shop owners who want to work with their own Piwik installation can be stored in the database through the POST request with following request payload. As the e-shop owner will most likely have set up his e-shop in his Piwik already, most of the parameters can be left blank:
(With example values.)

Key Description Value
ownerId The id of the e-shop owner in the process engine 1
piwikSiteName - left blank
piwikMainUrl - left blank
piwikLoginUsername - left blank
piwikLoginPassword - left blank
piwikUserEmail - left blank
piwikTimezone - left blank
piwikCurrency - left blank
piwikUserAlias - left blank
userPiwikSiteId Site id of the e-shop in the e-shop owner's Piwik 3
piwikTrackerLocationUrl URL to the Piwik installation of the e-shop owner www.my-shop.com/piwik
piwikAuthToken The Piwik auth_token of the e-shop owner's account 0123456789abcdef0123456789abcdef

Limitations of the service

With which parameters does the service scale?

How many concurrent E-Shops, how many concurrent products and how many users/E-Shop customers are possible without causing loss in quality/speed for the hardware described above?

In order to analise the visitor behaviour the engine needs to receive click events from Piwik (E-Compass Piwik Plugin). For the case that Piwik is connected to our Apache Server and uses SLL (https://e-compass.iao.fraunhofer.de/processengine/), load tests showed that the Engine is able to receive up to 200 click events/sec with an avg. processing time < 0.5 sec and a max. < 5 sec. If you connect to the Jetty Server directly (https://e-compass.iao.fraunhofer.de:8080/processengine/) the Engine is able to receive up to 300 click events/sec with an avg. processing time < 0.1 sec and a max. < 1 sec (300 click events/sec correspond to 1,200 new shop visitor per sec, staying 1 min and doing 15 clicks). The limit is about 360 click events/sec which can cause processing times above 10 sec (10sec is the maximum time for a voucher to work properly).
Parameters relevant for the scaling of this module are the overall number of users of all E-Shops and the number of rules. Each rule generates one request every ten minutes, i.e. the rules are checked every ten minutes. Each request takes 100 - 200 ms of time. So in ten minutes a maximum of 3000 request should be possible without further parallelization. There is one special kind of rule, which is the voucher rule (i.e. if a user shows a certain behaviour on the E-Shop a voucher is shown to him). This rule generates traffic depending on user behaviour, which means that this is not predictable. Therefore the above estimate can only give an orientation, not a guaranteed limit and one should always keep reserves for situations in which many users (over all E-Shops) show a behaviour that triggers a voucher rule.
If higher scaling was wanted, which of the hardware parameters would need to be increased?
The number of CPUs/CPU cores is the most critical. Eventually RAM and Storage may need to be increased also.
What else would be adjusted for higher scalability?
In order to make use of more CPU cores the software code would need to be parallelized, which is currently not the case.
Which further configuration would be necessary?
…

Contact Information Notification & Actions Engine

Florian Maier, florian.maier@iao.fraunhofer.de, +49 711 970-5121

Christopher Ruff, christopher.ruff@iao.fraunhofer.de, +49 711 970-2402

Retrieved from "https://websites.fraunhofer.de/E-COMPASS_Info_Guide/index.php?title=Notification_Actions_Engine&oldid=396"