Knowledge Base & Community Wiki
Data Management Fundamentals
Introduction – VisualizeIT offers three different sets of techniques for purposes of modelling and forecasting i.e. Analytical or Mathematical Modelling, Statistical Modelling an Simulation Modelling. Of the three modelling techniques Statistical Modelling requires the use of empirical data gathered from various systems whose behavior you intend to model for purposes of forecasting. While this article will not go into depth into the collection of empirical data for purposes of Statistical Modelling what we intend to do is look at some of the fundamental concepts surrounding upload of that empirical data into VisualizeIT for use with the various Statistical Modelling techniques.
- To learn more about the various supported data formats please visit Data Formats Supported by VisualizeIT.
- To obtain a set of input file formats with data stored in different frequencies please visit Sample Input Data Files.
Key Data Management Concepts – We can understand that Visualization, Modelling and Forecasting can be slightly intriguing and mesmerizing especially to someone who is just starting to look at the subject. To make the concepts easier to understand we have put together a workflow for Data Management which can be found here – Data Management Workflow. The Data Management Workflow is quite self explanatory and offers a step by step approach to successfully uploading the Time Series (empirical) data you have collected from your various systems into VisualizeIT for purposes of use with the Statistical Modelling techniques.
With all of that in the background let’s now look at a few terms unique to VisualizeIT that you ought to understand i.e. Applications and Data Dimensions.
- Applications – Applications are logical entities within VisualizeIT which can be associated with multiple Data Dimensions within VisualizeIT. There are no limits on the number of Applications or the number of Data Dimensions one can create within a given Application. A user however cannot delete an Application when corresponding Data Dimensions exist for a given Application. A user also cannot delete an Application if data exists in a given Data Dimension which is associated with a given Application.
- Data Dimensions – Data Dimensions are logical entities within VisualizeIT which should be used to upload empirical data (Time Series) into the system for use with the Statistical Models. Data Dimensions are associated with only one Application. Data Dimensions also cannot be deleted as long as there exists data for a given Data Dimension. A list of the various supported data formats can be found here – Supported Input Time Series Data Formats.
Accessing the Data Management Functionality – On logging into VisualizeIT please click on the “Statistical Models” link on the left hand side menu. This will cause the menu to drop down and a host of links revealed. Now click on the “Data Management” menu and you should see the screen below. The “Applications” tab part of the “Data Management” functionality within VisualizeIT allows the creation of logical “Applications” which are entities that multiple “Data Dimensions” are associated with. One “Application” can only be associated with one “Data Dimension” but many “Data Dimensions” can be associated with one “Application”.
Once you have successfully created an “Application” using the functionality provided by “Data Management” you can now proceed to the “Data Dimensions” tab. The “Data Dimensions” tab within VisualizeIT allows you to create multiple logical entities called “Data Dimensions” which eventually you would use to upload Time Series data into VisualizeIT. One “Application” can only be associated with one “Data Dimension” but many “Data Dimensions” can be associated with one “Application”.
Once you have created “Applications” and “Data Dimensions” it’s finally time to upload data into VisualizeIT. To upload data into VisualizeIT you select the “Data Import & Export” tab, then click on the relevant “Data Dimension” and finally upload Time Series data (from among the list of supported file formats) into VisualizeIT. Once data has been uploaded into the “Data Dimension” you are now ready to use the Statistical Modelling techniques for purposes of Visualization, Modelling & Forecasting.
The final piece of functionality within the “Data Management” page provides you the capability for Search, Edit and Delete of Time Series Data. This piece of functionality is aimed and helping users search for specific Time Series values that might be incorrect and then edit them or delete them from the given Data Dimension.
Conclusion – In a nutshell the Data Management capability within VisualizeIT is aimed at helping you create Applications, associate Data Dimensions with those Applications and finally upload data for the given Data Dimension into VisualizeIT. With data successfully uploaded into VisualizeIT you are then able to use the various statistical modelling techniques for purposes of Visualization, Modelling and Forecasting.
Modelling Solution: VisualizeIT offers access to a bunch of Analytical Models, Statistical Models and Simulation Models for purposes of Visualization, Modelling & Forecasting. Access to all the Analytical (Mathematical) models is free. We recommend you try out the Analytical models at VisualizeIT which are free to use and drop us a note with your suggestions, input and comments. You can access the VisualizeIT website here and the VisualizeIT modelling solution here –VisualizeIT.