Data Analyser

Physical Hardware Characteristics

Model:
Processor:
RAM:
Hard Drive Space:
Network Connection:
Hypervisor used:
Physical Load Balancing:

Virtual Machine Hardware Specifications and Operating System

Guest Operating System:
Processor:
RAM:	GB
Hard Drive Space VM:
Network Connection:
Minimum required Network Connection:

Service Environment and Set-up on VM1

Required Software

Software	Download
Software-Name	Software-Link
Software-Name	Software-Link
Software-Name	Software-Link

Software Licenses

Please indicate if a commercial provider would need to buy commercial licenses of a certain software used for operating the service and – if so – what cost this may produce approximately

OS Environment Variables

…

Installation

This following chapter provides detailed explanations for different technical modules developed by CICtourGUNE for the Data Analyser Module within the E-Compass Data Mining Services.

The Following subsections are described:

• Database
• REST services in JAVA
• Data Mining in R

In order to have a better understanding of the different developments, their location, distribution and used protocols are commented.

DATABASE

First of all, a MySQL or SQL Server database must be configured so that REST services an R data mining processes can both properly work. Currently those two database types are supported in both JAVA REST services and R processes, but other database types could be considered (if necessary).

1. MySQL general schema

2. SQL Server general schema

In the following lines, required database schema and tables will be described.  

MySql

After creating a new database in a MySQL server, those are needed code sentences in order to create the required tables:

1. E-shops table

Figure 1: E-Shops table CREATE code for MySQL

2. Competitors table

Figure 2: Competitors table CREATE code for MySQL

3. Products table

Figure 3: Products table CREATE code for MySQL

4. Visitors table

Figure 4: Visitors table CREATE code for MySQL  

SQL Server

After creating a new database in a SQL Server server, those are needed code sentences in order to create the required tables: 1. E-shops table

Figure 5: E-Shops table CREATE code for SQL Server

2. Competitors table

Figure 6: Competitors table CREATE code for SQL Server

3. Products table

Figure 7: Products table CREATE code for SQL Server

4. Visitors table

Figure 8: Visitors table CREATE code for SQL Server

REST SERVICES IN JAVA

Following lines describe existing REST methods developed by CICtourGUNE.

Rest services provided by CICtourGUNE were served in the following base URL address:

• http://ecompass.tourgune.org/...

However, having transferred technical infrastructure to UMA, this will be the final base URL address:

• http://ecompass.khaos.uma.es/...

Project distribution

Java project for REST services is distributed in the following packages:

1. Bean: Contains java classes to manage elements with different properties, for example an e-shop

a.	“Eshop.java”: Contains some variables to manage information related to an element of type “Eshop”

 i.	int eshopID

 ii.	String name 

 iii.	String url

 iv.	String currency

 v.	String chamber

 vi.	String analytics

 vii.	String ecommerceFunc

 viii.	boolean newEshop

2. Dao: Contains java methods to connect and interact with different databases. Currently “SqlServer” and “MySQL” databases are supported.

3. Math: Contains different mathematical operations required to calculate elasticity over competitors (HolisticCompetitor)

4. Rest API: Contains code that allows working with JSON format. In addition, following REST methods are defined (they are detailed in the following section):

 a.	Get data from existing e-shop for given date period and metric (GET)

 b.	Add new e-shop (POST)

 c.	Update info of existing e-shop (PUT)

 d.	Delete existing e-shop (PUT)

5. Utils: Contains global variables for common use in the whole JAVA project. Thus, variables affecting to different places in the project can more easily be changed in a single place.

Each “*.java” file is internally documented so that UMA can understand what has been developed or where make any change (if needed).

Figure 9: JAVA project for REST services

Description of REST methods

Get data from existing e-shop for given date period and metric (GET)

Base URL: http://ecompass.khaos.uma.es/ecompassAPI/getDataAnalyzerInfo/results.json

• Variables:

metric: “visitors”, “products” o “competitors”

eshopid: e-shop ID to get data from

startdate: start date threshold for data collection

enddate: end date threshold for data collection

dbtype: “MySQL” or “SqlServer”

• Pseudo-URL:

http://ecompass.khaos.uma.es/ecompassAPI/getDataAnalyzerInfo/results.json?metric=<metric>&startdate=<yyyy-mm-dd>&enddate=<yyyy-mm-dd>&eshopid=<eshopid>&dbtype=<dbtype>

• URL example:

http://ecompass.khaos.uma.es/ecompassAPI/getDataAnalyzerInfo/results.json?metric=visitors&startdate=2015-06-01&enddate=2015-06-10&eshopid=168&dbtype=MySQL

  Add new e-shop (POST)

Base URL: http://ecompass.khaos.uma.es/ecompassAPI/getDataAnalyzerInfo/notifyNewEshop

• Variables:

dbtype: “MySQL” or “SqlServer”

eshopid: new e-shop’s ID number

name: new e-shop’s name

url: new e-shop’s URL address

• Pseudo-URL:

http://ecompass.khaos.uma.es/ecompassAPI/getDataAnalyzerInfo/notifyNewEshop?dbtype=<dbtype>&eshopid=<eshopid>&name=<name>&url=<url>

• URL example:

http://ecompass.khaos.uma.es/ecompassAPI/getDataAnalyzerInfo/notifyNewEshop?dbtype=MySQL&eshopid=555&name=hugo&url=mikel

Update info of existing e-shop (PUT)

Base URL: http://ecompass.khaos.uma.es/ecompassAPI/getDataAnalyzerInfo/updateEshop

• Variables (en el path):

{dbtype}: “MySQL” or “SqlServer”

{eshopid}: e-shop ID to be updated

• Pseudo-URL:

http://ecompass.khaos.uma.es/ecompassAPI/getDataAnalyzerInfo/updateEshop/{dbtype}/{eshopid}

• URL example:

http://ecompass.khaos.uma.es/ecompassAPI/getDataAnalyzerInfo/updateEshop/MySQL/999

PAYLOAD (needed additional content to be sent in JSON format):

• Variables:

eshopID: new e-shop ID for the e-shop in URL address (999 in the example)

name: new e-shop name

url: new e-shop URL address

• PAYLOAD example:

{"eshopID":171,"name":"Coozina","url":"http://www.coozina.gr"}

Delete existing e-shop (PUT)

Base URL: http://ecompass.khaos.uma.es/ecompassAPI/getDataAnalyzerInfo/deleteEshop

• Variables (in the path):

{dbtype}: “MySQL” or “SqlServer”

{eshopid}: existing e-shop’s ID

{name}: e-shop name to be deleted, corresponding to “eshopid” variable

• Pseudo-URL:

http://ecompass.khaos.uma.es/ecompassAPI/getDataAnalyzerInfo/deleteEshop/{dbtype}/{eshopid}/{name}

• URL example:

http://ecompass.khaos.uma.es/ecompassAPI/getDataAnalyzerInfo/deleteEshop/MySQL/999/Coozina

 

DATA MINING IN R

Following lines describe required files for using data mining processes in R developed by CICtourGUNE. File Description Following files are the ones existing in the R Project: CODE directory

Preparation

…

Service Configuration

…

Configuration script

availability / location

README / User Manual

availability / location

Configuration steps

… Configuration of REST endpoints at: …

Operation

Service startup procedure

…

Restarting the service

…

Service Logs

…

Recurring Manual Actions / Maintenance Tasks

…

Other

…

Limitations of the service

With which parameters does the service scale?

How many concurrent E-Shops, how many concurrent products and how many users/E-Shop customers are possible without causing loss in quality/speed for the hardware described above?

…

If higher scaling was wanted, which of the hardware parameters would need to be increased?

…

What else would be adjusted for higher scalability?

…

Which further configuration would be necessary?

…

Contact Information Data Analyser

José Manuel García Nieto, jnieto@lcc.uma.es, +34 951 952924