Nsys Cloud Warehouse
The trend of the last years is in area of cloud computing based on various services which are related to one or more particular fields such as Software as a Service (SaaS), Infrastructure as a Service (IaaS) and Platform as a Service (PaaS).
The cloud services are operated by different providers which are usually oriented in specific field such as storage or computing. The common requirement from the users of cloud services is to have an option for getting reports and analyses tasks helping them to get an overview about cloud services usage. The challenge in the terms of reporting and analyses is in approach of collecting data from the various systems, store them somewhere and generate reports with some results. The ideal scenario would be in use of a traditional data warehouse which is capable to collect data from the operational systems and pass the data through an operational data store for additional operations before they are used in the warehouse for reporting. Unfortunately the traditional data warehouse implementations are too complex for flexible and easy way how to extend their functionality and mostly oriented for some particular solutions of the 3rd parties. There is missing a way how to do a simple integration with other systems. The Nsys Cloud Warehouse presents another view on data warehouse focused on data of the cloud services.
The cloud warehouse is a database used for reporting, analyses and data synchronization. The data stored in the cloud warehouse are collected from the external/operational systems provided as cloud services. The data may pass through an operational data store (ODS) for additional operations before it is used in the cloud warehouse for reporting. The cloud warehouse maintains its function on three layers: collecting, integration, and access. Collecting is used to store raw data from external/operational systems. The integration layer is used to integrate data and to have a level of abstraction from users. The access layer is for getting data out for users. This is definition of the cloud warehouse focused on data storage of cloud services.
Figure 1 - Nsys Cloud Warehouse Infrastructure
The collecting and integration layers are implemented as a plugin for Nsys Daemon which represents the main system service responsible for the network node management. The access layer is implemented as an extension for the Nsys Portal which is providing user interface and user management for the cloud warehouse.
Firstly is implemented collector which is responsible for the collecting data from external/operational systems. The interaction with both systems is implemented through a provider which is responsible for connection to remote system and collects selected data. There can be one provider which connects for instance to Dropbox and another which connects to Amazon S3 and collects data there. Each provider maps data from remote cloud service to cloud warehouse through Nsys Query Language (NQL).
Secondly is implemented a system for integration layer and operational data store. The data are stored by collector in a buffer which is used by operational data store for additional operations. The ODS can pass data back to the external/operational system for further operations and to the cloud warehouse. Data in cloud warehouse can be stored for a limited time or without any limits.
An access layer is implemented as last one, and it is allowing operate with cloud ware-house reports.