SICDB Doc - User contributions [en]

Version

2024-07-12T05:44:32Z

Salkin: /* Version Information */

== Version Information ==

The SICdb dataset uses a versioning system. It contains 3 numbers major, minor and patch. A suffix may be added for changes, the do not necessarily needs to be applied.

A major version change generally occurs when the database was incrementally updated and more cases are included.

A minor version change occurs when data was altered that will likely change most study data. (i.e. removal of a significant amount of invalid cases) A minor update also applies when new data is added (i.e. a new table) but the existing data is not altered.

A patch version change occurs when only specific data was altered and will not affect all exports. Please read changelog to check if your data is affected.

=== 1.0.8 (07/2024) ===
* Added LOINC codes to laboratory references

=== 1.0.7 (04/2024) ===
* Added field `cases`.`HospitalDischargeDay` `HospitalStayDays` representing the day of release from hospital after admission and the full lenght of hospital stay, respectively.
* Added field `cases`.`AdmissionUrgency`, depicting the urgency of admission
* Added High Flow (HFNC) therapy data
* Added Richmond Agitation-Sedation Scale (RASS) score
* Added Numeric Rating Scale (NRS-11)
* Added SOFA Score
* Removed 36 invalid cases
* Recalculated field `cases`.`OffsetAfterFirstAdmission`, fixing an issue that occasionally led to inadequate values

=== 1.0.6 (05/2023) ===

* [Important] Update a major error at the heart rate signal (invalid mapping of ecg signal)
* [Important] Renamed heart rate signals
* [Important] Updated some (~150) invalid weight/height signals
* [Fix] Due to a change in table structure the Premedication_ fields missed reference id, fixed
* Added KDIGO_AKI_168 and published algorithm (note: in general be careful with using urine output KDIGO in automated datasets)
* Added field ICUOffset for better comparability with other ICU datasets
* Added more signal data
* Corrected a small issue in the norepinephrine per kg algorithm (change is not significant)

=== 1.0.5 ===

* Merged additional mortality data and changed OffsetOfDeath from in-hospital-mortality to general 1-year-mortality
* Removed or fixed 190 cases of inplausible height or weight data
* Added some more CRRT data
* Recalculated DrainageSum

=== 1.0.4 ===

* Added unity of measurements for signal data
* Structural changes

Data Description Cases

2024-07-12T05:42:21Z

Salkin:

"The "cases" table is the base table of the SICdb dataset and the "CaseID" field serves as the identifier that relates all data. Each admission to the intensive care unit generates a unique "CaseID." To identify readmissions, a "PatientID" is provided. The "OffsetAfterFirstAdmission" field stores the time between the first admission and the current one.

"TimeOfStay" is the time in seconds from the admission recorded by the MetaVision system to the discharge. This time may include any preceding surgery and ends when the case is closed.

There are three fields indicating survival: "DischargeState" indicates the patient's state upon discharge from the ICU and is taken from the MetaVision discharge form. "HospitalDischargeType" provides information on how the case was closed in the clinical information system. "OffsetOfDeath" is the time in seconds from admission to death and includes additional hospital stays and out of hospital mortality. As it refers to a 1-year mortality it is set to null if death occurred more than one year after admission.

{| class="wikitable"
|+ cases.csv.gz
|-
! Name !! Type !! Description !! Comment
|-
| CaseID || Integer || A randomly assigned identifier || CaseID is unique to each admission.
|-
| PatientID || Integer ||A randomly assigned identifier || PatientID is unique to each Patient, useful to identify readmission
|-
| AdmissionYear || Integer || Year of admission ||
|-
| TimeOfStay || Integer || Time of stay (seconds) || Time from primary metavision admission, to last discharge. In cases, where the patient is admitted to ward after surgery, this will include surgery time.
|-
| ICUOffset || Integer || Time of actual ICU admission || SICdb includes preceding surgery if applicable, this field indicates the first transfer to an intensive or intermediate care ward
|-
| saps3 || Float || Simplified Acute Physiology Score III ||
|-
| HospitalDischargeType || Reference || Type of hospital discharge. || This field indicates survival.
|-
| HospitalDischargeDay || Integer || The day of hospital discharge after admission || Note that this data is only available in days
|-
| HospitalStayDays || Integer || Days stayed in hospital for this case, including pre icu. ||
|-
| DischargeState || Reference || Type of icu discharge || This field indicates survival on icu
|-
| DischargeUnit || Reference || Unit the patient was discharged to, as selected in Metavision discharge form ||
|-
| OffsetOfDeath || Integer || 1-year mortality in seconds from primary admission to death || 1 year mortality, including out-of-hospital as far as known. See {{:Mortality}} for further information.
|-
| EstimatedSurvivalObservationTime || Reference || Estimation of oversation time, either 1-year or 6-month||
|-
| Sex || Reference || ||
|-
| WeightOnAdmission || Float || Rounded to +-5kg ||
|-
| HeightOnAdmission || Float || Rounded to +-5cm ||
|-
| AgeOnAdmission || Integer || Rounded to +-5y, over 90 set to 90 ||
|-
| HospitalUnit || Reference || Last unit using this case ||
|-
| ReferringUnit || Reference || Referring unit as selected in admission form || Note: Unfortunatly in some cases "Notaufnahme" is selected here, so the referring unit is not specified in these cases.
|-
| ICD10Main || Text || ICD10 main code ||
|-
| ICD10MainText || Text || ICD10 main text ||
|-
| SurgicalSite || Reference || ||
|-
| InterventionsText || Text || List of interventions ||
|-
| HoursOfCRRT || Integer || Hours of continuous renal replacement threapy this admission **||
|-
| AdmissionFormHasSepsis || Reference || A mandatory field in the admission form **||
|-
| HeartSurgeryAdditionalData || For heart surgery patients there is additional data collected **|| Yes if applicable
|-
| HeartSurgeryCPBTime || Integer || Bypass time **||
|-
| HeartSurgeryBeginOffset || Integer || Offset in seconds from ICU admission to cut **||
|-
| HeartSurgeryEndOffset || Integer || Offset in seconds from ICU admission to end of surgery** ||
|-
| OffsetAfterFirstAdmission || Integer || If a patient has more than one admission, this is the offset in seconds from the first ||

|}

* These fields are not available on PhysioNet at the moment. Contact us for further information.
** These fields will be moved to data tables in version 1.1.0

Introduction

2024-07-12T05:41:29Z

Salkin:

The SICdb dataset provides insight to over 27 thousand intensive care admissions, including therapy and data of their preceding surgery. Data was collected between 2013 and 2021 from 4 of the intensive care units at the University Hospital Salzburg, having more than 3 thousand intensive care admissions per year on 41 beds. The dataset is deidentified and contains, amongst others, case information, laboratory, medication, monitor and respirator signal data.SICdb provides aggregated once-per-hour and highly granular once-per-minute data.

The dataset, version 1.0.8, includes data from more than 27,350 admissions to the Department of Anesthesiology and Intensive Care Medicine at the General Hospital Salzburg and Paracelsus Medical University.

== Data Tables ==
The SICdb dataset consists of billions of data entries across 7 data tables. The main table, "cases," contains a single entry for each intensive care admission and includes information about the patient (such as age, weight, and sex) and case details (such as diagnosis, scores, and ICD10 codes). The "TimeOfStay" field indicates the time from the first admission to a Metavision-enabled ward to the final closing of the case, including any preceding surgery. The "OffsetOfDeath" field indicates survival in seconds from admission. This data represents in-hospital mortality and may extend beyond the length of the intensive care stay or the current hospital stay. Personal data such as age, weight, and height have been grouped into bins of 5, with ages over 90 placed in the final bin.

All other data tables are related to the "cases" table through the "CaseID" field. Most data has timing information, with the "offset" field indicating the number of seconds from admission to the time of the event. The "laboratory" table contains laboratory values and the "medication" table provides data on administered drugs. There are several generic data tables that contain data sorted by type. The "data_ref" table contains additional nominal/categorical data, one entry per admission, and the "data_range" table documents items with a start and end time, such as data on central lines or drainages. The "data_float_h" table contains float data, aggregated once per hour, and includes most signal data. To reduce table size, minute data is serialized as a stream of IEEE 754 floats in the "data_float_h.rawdata" field. For further instructions on using this data, see the documentation found in Documentation.pdf or online [7], and refer to the unpacking script example provided on our GitHub repository [8].

== Reference Table ==
Nominal data is encoded, and the reference table "d_references" provides additional information about the associated field. The referenced fields in all data tables correspond to the primary key of the "d_references" table, "ReferenceGlobalID." The "ReferenceValue" field in "d_references" gives the variable's value, and the "ReferenceUnit" field holds the unit or measurement, if applicable.

== Data Format ==
GZip-compressed RFC 4180 comma-separated files are provided. The most current documentation, including table schemas, can be found online , and an offline copy is included in the files under the name Documentation.pdf. A GitHub repository has been created to share code, report issues, and discuss the dataset.

Data Description References

2024-07-12T05:41:00Z

Salkin:

The d_references table contains information on all encoded data fields of the SICdb dataset. Each field, that has "Reference" as field type, is associated with the ReferenceGlobalID in the d_references table. Additionally ReferenceUnit describes the unit of measurement used for this field. Refer to chapter [[SQL Examples]] to learn how to easily use this table in relational databases.

{| class="wikitable"
|+ d_references.csv.gz
|-
! Name !! Type !! Description !! Comment
|-
| ReferenceGlobalID || Integer || The unique ID for the reference || Use this identifier as dictionary for alle encoded fields
|-
| ReferenceValue || Text || Reference value || i.e. "Creatinine"
|-
| ReferenceName || Text || The name of the reference || i.e. "Laboratory"
|-
| ReferenceUnit || Text || The unit of this item if applicable || i.e. "mg/dl"
|-
| LOINC_code || Text || LOINC Code ||
|-
| LOINC_short || Text || LOINC SHORTNAME ||
|-
| LOINC_long || Text || LOINC LONG_COMMON_NAME ||

|}

Data Description References

2024-07-12T05:39:48Z

Salkin:

Version

2024-04-05T08:53:01Z

Salkin: /* Version Information */

== Version Information ==

The SICdb dataset uses a versioning system. It contains 3 numbers major, minor and patch. A suffix may be added for changes, the do not necessarily needs to be applied.

A major version change generally occurs when the database was incrementally updated and more cases are included.

A minor version change occurs when data was altered that will likely change most study data. (i.e. removal of a significant amount of invalid cases) A minor update also applies when new data is added (i.e. a new table) but the existing data is not altered.

A patch version change occurs when only specific data was altered and will not affect all exports. Please read changelog to check if your data is affected.

=== 1.0.7 (04/2024) ===
* Added field `cases`.`HospitalDischargeDay` `HospitalStayDays` representing the day of release from hospital after admission and the full lenght of hospital stay, respectively.
* Added field `cases`.`AdmissionUrgency`, depicting the urgency of admission
* Added High Flow (HFNC) therapy data
* Added Richmond Agitation-Sedation Scale (RASS) score
* Added Numeric Rating Scale (NRS-11)
* Added SOFA Score
* Removed 36 invalid cases
* Recalculated field `cases`.`OffsetAfterFirstAdmission`, fixing an issue that occasionally led to inadequate values

=== 1.0.6 (05/2023) ===

* [Important] Update a major error at the heart rate signal (invalid mapping of ecg signal)
* [Important] Renamed heart rate signals
* [Important] Updated some (~150) invalid weight/height signals
* [Fix] Due to a change in table structure the Premedication_ fields missed reference id, fixed
* Added KDIGO_AKI_168 and published algorithm (note: in general be careful with using urine output KDIGO in automated datasets)
* Added field ICUOffset for better comparability with other ICU datasets
* Added more signal data
* Corrected a small issue in the norepinephrine per kg algorithm (change is not significant)

=== 1.0.5 ===

* Merged additional mortality data and changed OffsetOfDeath from in-hospital-mortality to general 1-year-mortality
* Removed or fixed 190 cases of inplausible height or weight data
* Added some more CRRT data
* Recalculated DrainageSum

=== 1.0.4 ===

* Added unity of measurements for signal data
* Structural changes

2023-01-06T09:54:07Z

Salkin:

The SICdb dataset provides insight to over 27 thousand intensive care admissions, including therapy and data of their preceding surgery. Data was collected between 2013 and 2021 from 4 of the intensive care units at the University Hospital Salzburg, having more than 3 thousand intensive care admissions per year on 37 beds. The dataset is deidentified and contains, amongst others, case information, laboratory, medication, monitor and respirator signal data.SICdb provides aggregated once-per-hour and highly granular once-per-minute data.

The dataset, version 1.0.4, includes data from more than 27,386 admissions to the Department of Anesthesiology and Intensive Care Medicine at the General Hospital Salzburg and Paracelsus Medical University.

== Data Tables ==
The SICdb dataset consists of billions of data entries across 7 data tables. The main table, "cases," contains a single entry for each intensive care admission and includes information about the patient (such as age, weight, and sex) and case details (such as diagnosis, scores, and ICD10 codes). The "TimeOfStay" field indicates the time from the first admission to a Metavision-enabled ward to the final closing of the case, including any preceding surgery. The "OffsetOfDeath" field indicates survival in seconds from admission. This data represents in-hospital mortality and may extend beyond the length of the intensive care stay or the current hospital stay. Personal data such as age, weight, and height have been grouped into bins of 5, with ages over 90 placed in the final bin.

All other data tables are related to the "cases" table through the "CaseID" field. Most data has timing information, with the "offset" field indicating the number of seconds from admission to the time of the event. The "laboratory" table contains laboratory values and the "medication" table provides data on administered drugs. There are several generic data tables that contain data sorted by type. The "data_ref" table contains additional nominal/categorical data, one entry per admission, and the "data_range" table documents items with a start and end time, such as data on central lines or drainages. The "data_float_h" table contains float data, aggregated once per hour, and includes most signal data. To reduce table size, minute data is serialized as a stream of IEEE 754 floats in the "data_float_h.rawdata" field. For further instructions on using this data, see the documentation found in Documentation.pdf or online [7], and refer to the unpacking script example provided on our GitHub repository [8].

== Reference Table ==
Nominal data is encoded, and the reference table "d_references" provides additional information about the associated field. The referenced fields in all data tables correspond to the primary key of the "d_references" table, "ReferenceGlobalID." The "ReferenceValue" field in "d_references" gives the variable's value, and the "ReferenceUnit" field holds the unit or measurement, if applicable.

== Data Format ==
GZip-compressed RFC 4180 comma-separated files are provided. The most current documentation, including table schemas, can be found online [7], and an offline copy is included in the files under the name Documentation.pdf. A GitHub repository has been created to share code, report issues, and discuss the dataset [8].

Data Description Cases

2022-12-13T08:04:55Z

Salkin:

The cases table is the base table of the SICdb dataset, '''CaseID''' is the identifier that relationally connects all data. For each admission to our intensive care unit one CaseID is generated. To identify readmissions, a PatientID is provided. The time between the first admission and the current is stored in field OffsetAfterFirstAdmission.

TimeOfStay is the time in seconds from Metavision admission to Metavision discharge. This time may begin with the preceding surgery, and ends when the case is closed.

There are 3 fields indicating survival. DischargeState indicates the state of the patient on ICU discharge, this data is taken from the Metavision discharge form. HospitalDischargeType gives information about how this case was closed in the clinical information system. OffsetOfDeath is the time in seconds from admission to death. This field also takes further stays into consideration and is, opposite to DischargeState and HospitalDischargeType, not linked to the stay identifier. Take into consideration that OffsetOfDeath cannot be used as long-term survival information, because it will only be set if the patient died within a stay in hospital.

{| class="wikitable"
|+ cases.csv.gz
|-
! Name !! Type !! Description !! Comment
|-
| CaseID || Integer || A randomly assigned identifier || CaseID is unique to each admission.
|-
| PatientID || Integer ||A randomly assigned identifier || PatientID is unique to each Patient, useful to identify readmission
|-
| AdmissionYear || Integer || Year of admission ||
|-
| TimeOfStay || Integer || Time of stay (seconds) || Time from primary metavision admission, to last discharge. In cases, where the patient is admitted to ward after surgery, this will include surgery time.
|-
| saps3 || Float || Simplified Acute Physiology Score III ||
|-
| HospitalDischargeType || Reference || Type of hospital discharge. || This field indicates survival.
|-
| DischargeState || Reference || Type of icu discharge || This field indicates survival on icu
|-
| DischargeUnit || Reference || Unit the patient was discharged to, as selected in Metavision discharge form ||
|-
| OffsetOfDeath || Integer || Seconds from primary admission to death || Not linked to current stay, may include normal ward and later stays
|-
| Sex || Reference || ||
|-
| WeightOnAdmission || Float || Rounded to +-5kg ||
|-
| HeightOnAdmission || Float || Rounded to +-5cm ||
|-
| AgeOnAdmission || Integer || Rounded to +-5y, over 90 set to 90 ||
|-
| HospitalUnit || Reference || Last unit using this case ||
|-
| ReferringUnit || Reference || Referring unit as selected in admission form || Note: Unfortunatly in some cases "Notaufnahme" is selected here, so the referring unit is not specified in these cases.
|-
| ICD10Main || Text || ICD10 main code ||
|-
| ICD10MainText || Text || ICD10 main text ||
|-
| DiagnosisT1 || Text || Diagnosis field 1* ||
|-
| DiagnosisT2 || Text || Diagnosis field 2 ||
|-
| DiagnosisT3|| Text || Diagnosis field 3* ||
|-
| SurgicalSite || Reference || ||
|-
| InterventionsText || Text || List of interventions ||
|-
| HoursOfCRRT || Integer || Hours of continuous renal replacement threapy this admission || Note: This field will be moved into data table soon
|-
| AdmissionFormHasSepsis || Reference || A mandatory field in the admission form ||
|-
| HeartSurgeryAdditionalData || For heart surgery patients there is additional data collected || Yes if applicable
|-
| HeartSurgeryCPBTime || Integer || Bypass time ||
|-
| HeartSurgeryBeginOffset || Integer || Offset in seconds from ICU admission to cut ||
|-
| HeartSurgeryEndOffset || Integer || Offset in seconds from ICU admission to end of surgery ||
|-
| OffsetAfterFirstAdmission || Integer || If a patient has more than one admission, this is the offset in seconds from the first ||

|}

* These fields are not available on PhysioNet at the moment. Contact us for further information.

Data Description Cases

Salkin: /* Data Files */

The SICdb dataset consists from 7 files. Additional data is found on our [https://github.com/nrodemund/sicdb github repository], refer to [[Get Started]] for usage details.

== Data Files ==

All data files are gzip compressed RFC4180 comma seperated files.

{| class="wikitable"
|+ Files
|-
! Name !! Short description
|-
| [[Case Data|cases.csv.gz]] || Contains basic data about admissions to icu. See [[Case Data]] for table details and [[Case Description]].
|-
| [[d_references|d_references.csv.gz]] || Contains the dictionary for all encoded fields in the SICdb dataset. ||
|-
| [[data_float_h|data_float_h.csv.gz]] || Contains hourly aggregated float data like monitor [[Signal Data|signals]] or respirator settings. Additionally raw minute data is included.See [[data_float_h]] and [[Signal Data]] for further information. ||
|-
| [[data_ref|data_ref.csv.gz]] || Contains encoded (nominal) data related to an admission. ||
|-
| [[Laboratory Data|laboratory.csv.gz]] || Contains laboratory results. ||
|-
| [[Medication Data|medication.csv.gz]] || Contains data about medication given during ICU stay. ||
|-
| [[Data range|data_range.csv.gz]] || Contains ranged data (from/to) ||
|-
| [[Unitlog|unitlog.csv.gz]] || Information about the patient's stay ||
|}

== Configuration Files ==

This data is found on our [https://github.com/nrodemund/sicdb github repository].

=== docker-compose.yml ===

This file is needed to install our database software. See [[Software Description]] for further information, and [[Get Started]] for installation instructions.

=== DatasetConfiguration.conf ===

Our database software needs this metadata for building up the database. It contains field data types and SQL table structure, that cannot be saved in .csv files.

== RooData Folder and contents ==

This data is used by our software and contains field and table configurations.

Deserialize Raw Data

2022-11-09T10:26:51Z

Salkin:

<noinclude>== Why is this data serialized? ==</noinclude>
<includeonly>===Why is this data serialized? ===</includeonly>

The intermediate data, where minute values are not aggregated, has roughly about 240gb, and it is to be expected that is increases by about 15% in 2023 update. By aggregating and serializing these values the database can be compresssed to under 10gb. Nevertheless it complicates the deserialization of this data. While our software can export the minute values one-click, using it in raw data requires a bit of coding.

Refer to chapter Full Unpack to learn how to completely unpack minute data.

<noinclude>== Encoding ==</noinclude>
<includeonly>=== Encoding ===</includeonly>

The raw data field is a stream of 60 little endian IEEE 754 floats, so it has exactly 240 bytes. The first 4 bytes represent the first minute of the hour and so on. Note that 0x000000 is defined to be NULL (no value).

<noinclude>== Python example ==</noinclude>
<includeonly>=== Python example ===</includeonly>

def GetRawValues(data):
ret=[]
for i in range(int(len(data)/4)):
#if (data[i*4]==0 and data[i*4+1]==0 and data[i*4+2]==0 and data[i*4+3]==0): continue # uncomment this if you do not want to include null values
ret.append(struct.unpack('<f',data[i*4:i*4+4])[0])
return ret

This simple example does not include a correct offset for the values. You may check our [https://github.com/nrodemund/sicdb/tree/main/Scripts/Unpack%20raw%20data unpack script] to find an example on how to compute the offset.

<noinclude>== C# example ==</noinclude>
<includeonly>=== C# example ===</includeonly>

public static float?[] GetRawValues(byte[] data)
{
byte[] buf = new byte[4];
float?[] ret = new float?[data.Length / 4];
for(int i = 0; i < data.Length; i += 4)
{
buf[0] = data[i];
buf[1] = data[i+1];
buf[2] = data[i+2];
buf[3] = data[i+3];
if (buf[0] == 0 && buf[1] == 0 && buf[2] == 0 && buf[3] == 0) continue; // ignore null values
ret[i / 4] = BitConverter.ToSingle(buf); // note: if you are on a BigEndian machine you need to flip buf
}
return ret;
}

<noinclude>== Full Unpack ==</noinclude>
<includeonly>=== Full Unpack ===</includeonly>

We provide a simple unpack python script on our [https://github.com/nrodemund/sicdb/tree/main/Scripts/Unpack%20raw%20data github code repository].