Data Collector Consolidator
Contents
Overview
The process of translating the collected data from different sources to RFD is carried out by means of mapping functions. Each data source considered in the scope of the SME E-COMPASS project has a different method for collecting, gathering, and providing access to the analytical data. Therefore, a different set of mapping functions is required in order to parse the information provided by each data source to RDF, according to the ontology. Each set of mappings is then composed by functions to translate the attributes with their values into their corresponding triplet form in RDF. That is, for most of the attributes, a corresponding mapping function has to be developed, involving a given class in the ontology. Nevertheless, as a number of analytic attributes share a common structure in the ontology, they can be
Physical Hardware Characteristics
| Model: | |
| Processor: | |
| RAM: | |
| Hard Drive Space: | |
| Network Connection: | |
| Hypervisor used: | |
| Physical Load Balancing: |
Virtual Machine Hardware Specifications and Operating System
| Guest Operating System: | |
| Processor: | Dual 2 core Intel processor and i7 intel 5 cores |
| RAM: | 16 GB GB |
| Hard Drive Space VM: | 100 GB internal |
| Network Connection: | 1 Gbit/s Ethernet |
| Minimum required Network Connection: |
Service Environment and Set-up on VM1
The API of the system and the corresponding database are located on VM1. The API is based on the Flask Microframework for Python and running within an Apache Webserver . The database is a MySQL database . The interface is implemented in Python. For setting up the API please download and install the following software:
| Software | Download |
|---|---|
| Apache 2.4 | http://httpd.apache.org/ |
| Python 2.7 | https://www.python.org/download/releases/2.7/ |
| MySQL Server 5.6 (Community Edition) | https://dev.mysql.com/downloads/mysql/ |
| Mod_wsgi for Apache 2.4 and Python 2.7 | http://www.lfd.uci.edu/~gohlke/pythonlibs/#mod_wsgi |
Software Licenses
Please indicate if a commercial provider would need to buy commercial licenses of a certain software used for operating the service and – if so – what cost this may produce approximately Openlink Software Virtuoso Universal server (used as RDF repository in the E-Compass Data Mining Services) requires a software license, which is free of cost for academic use only. In order to run this software productively a commercial license is required. The terms of licensing are available here
OS Environment Variables
…
Installation
…
Preparation
…
Service Configuration
Configuration script
availability / location
README / User Manual
availability / location
Configuration steps
…
Configuration of REST endpoints at:
Operation
Service startup procedure
…
Restarting the service
…
Service Logs
…
Recurring Manual Actions / Maintenance Tasks
…
Other
…
Limitations of the service
- With which parameters does the service scale?
How many concurrent E-Shops, how many concurrent products and how many users/E-Shop customers are possible without causing loss in quality/speed for the hardware described above?
- We are currently managing 20 E-Shops with data for 2 months by average (~30GBs + 20GBs)
- If higher scaling was wanted, which of the hardware parameters would need to be increased?
- RAM and HD storage
- What else would be adjusted for higher scalability?
- …
- Which further configuration would be necessary?
- …
Contact Information Data Collector & Consolidator Service
José Manuel García Nieto, jnieto@lcc.uma.es, +34 951 952924
