Difference between revisions of "Get Started"

From SICDB Doc
Jump to navigation Jump to search
Tag: Reverted
 
(21 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Introduction ==  
== Introduction ==  


The SICdb dataset is provided in compressed .csv files, the minute values are even more consolidated. Refer to the [[Main Page]] for detailed description of files.
{{Beta Version}}


The SICdb dataset contains billions of entries, therefore building up a database may present a challenge. Therefore a 'as simple as possible' solution is provided. Our solution, we called it [[SICdb Environment]], provides a fully preconfigured and fast environment to access, explore and export SICdb data. Refer to the Quick Start chapter if you know how the commandline and docker is working, skip to the Detailed Instructions for a more detailed reference.
The SICdb dataset is provided in compressed .csv files, the minute values are even more consolidated. Refer to the [[File List]] for detailed description of files.  


== Quick Start ==
The SICdb dataset contains billions of entries, therefore building up a database may present a challenge.


The database can be built up using [http://www.docker.com Docker]. After install navigate into the folder containing all the data and run "docker compose up". When the environment is running open http://localhost:5000 to install the dataset. The provided environment ist fully preconfigured, just press start and wait. Due to the vast size this may take 2-8 hours*. When install is finished the server has to be restarted, you may do this by reloading the page and then press the shiny restart button.
== Usage Examples ==


*) We work on a solution to provide a fully indexed database. Until now we have not found a legally safe way of distribution (repository). The fully indexed database has about 60gb, when minute values are indexed it is more than 250gb.
Scripting or Database Query examples for using SICdb are found at https://github.com/nrodemund/sicdb/tree/main/Examples


== Detailed Instructions ==
== Relational Database Import ==
 
Refer to https://github.com/nrodemund/sicdb/tree/main/Import for some scripts for importing the dataset into a relational database system.

Latest revision as of 08:51, 5 April 2024

Introduction

SICdb dataset and the documentation, as of 04/24, in active development. We try to improve our project as fast as we can! Please contact us for every problem with data or software you find! We'd love to find some motivated researchers attain as much as possible from our dataset. Contact us if you face any issues!

The SICdb dataset is provided in compressed .csv files, the minute values are even more consolidated. Refer to the File List for detailed description of files.

The SICdb dataset contains billions of entries, therefore building up a database may present a challenge.

Usage Examples

Scripting or Database Query examples for using SICdb are found at https://github.com/nrodemund/sicdb/tree/main/Examples

Relational Database Import

Refer to https://github.com/nrodemund/sicdb/tree/main/Import for some scripts for importing the dataset into a relational database system.

Retrieved from "https://www.sicdb.com/wiki/index.php?title=Get_Started&oldid=380"