Create archive from a dataset
Creates an archive of a dataset - a snapshot of its data and metadata.
This operation requires space_manage_datasets and space_create_archives privileges.
Example cURL requests
Create archive from a dataset
curl -H "X-Auth-Token: $TOKEN" -X POST "https://$PROVIDER_HOST/api/v3/oneprovider/archives" \
-H "Content-Type: application/json" -d '{
"datasetId": "'$DATASET_ID'",
"config": {
"incremental": {"enabled": true},
"includeDip": false,
"layout": "bagit",
"createNestedArchives": true,
"followSymlinks": true
},
"preservedCallback": "https://example.org/preserved_archives",
"deletedCallback": "https://example.org/deleted_archives",
"description": "Archived dataset with experiment data from 2021."
}'
Request body
Dataset properties.
Id of the dataset that will be archived.
config Parameters used to configure the process of creating an archive.
Parameters used to configure the process of creating an archive.
incremental This field determines whether archive is incremental - storing only files that have
changed in comparison to the base archive (provided under the "basedOn" key).
Unchanged files are preserved as hard links to the corresponding files in the base archive.
If does not enabled, all files belonging to the dataset are stored directly in the archive.
This field determines whether archive is incremental - storing only files that have changed in comparison to the base archive (provided under the "basedOn" key). Unchanged files are preserved as hard links to the corresponding files in the base archive. If does not enabled, all files belonging to the dataset are stored directly in the archive.
This attribute determines which archive will be the base archive for incremental creation of a new archive
(consult description of the incremental parameter).
If not set, last successfully preserved archive will be used.
If incremental is not enabled this field is ignored.
This flag determines whether dissemination information package (DIP) is created alongside with archival information package (AIP), on the storage. For more information on DIP and AIP please consult Open Archival Information System.
The layout of files and directories in an archive.
plain- Structure of the data in the archive is an exact copy of the dataset.bagit- Data in the archive are stored in the bagit format.
This flag determines the strategy of archiving nested datasets (i. e. datasets inside the dataset that is being archived).
If set to true, a separate archive will be created for each nested dataset, and a symbolic link to the nested archive will
be created in the parent archive in the place of the nested dataset's root file/directory. The procedure is recursive and
may effectively create a tree of linked archives.
If set to false, the resulting archive will be monolithic, i. e. all files and directories from nested datasets will be copied
directly to the archive, in their respective locations. Effectively, the internal hierarchy of datasets has no impact on the
resulting archive content.
This flag determines whether symbolic links pointing to paths outside of the dataset should be resolved during archive creation.
If set to true, valid links will be resolved and their target files/directories will be
copied to the archive in their place. Invalid symbolic links (not resolvable to a valid path
in the space) will be ignored and not included in the archive.
If set to false, symbolic links will be copied to the resulting archive and their target
paths will not be modified. Note that these symbolic links may target modifiable files in the space.
NOTE: Symbolic links pointing to files inside the dataset are always preserved, regardless of this setting. Their target paths are reconstructed to point to the corresponding files in the resulting archive.
User defined description of the archive.
URL on which POST request will be performed to notify that process of persisting an archive has finished. The callback request will include body in the following JSON format:
{
"archiveId": "$ARCHIVE_ID",
"datasetId" : "$DATASET_ID",
"error": "Error description"
}
"error" field will have null value if archivisation has been finished successfully.
Requested server should response with one of 2xx HTTP codes. If the request fails or returns any
other HTTP code, it will be retried.
URL on which POST request will be performed to notify that process of purging an archive has finished. The callback request will include body in the following JSON format:
{
"archiveId": "$ARCHIVE_ID",
"datasetId": "$DATASET_ID",
"error": "Error description"
}
"error" field will have null value if purging of an archive has been finished successfully.
Requested server should response with one of 2xx HTTP codes. If the request fails or returns any
other HTTP code, it will be retried.
Request Examples
{
"datasetId": "1f4b762b1380946e73aeca574c77f14c",
"config": {
"incremental": {
"enabled": true,
"basedOn": "ae6f78c89a97c9e78e891105f703bcb8"
},
"includeDip": false,
"layout": "plain"
},
"preservedCallback": "https://example.org/preserved_archives",
"deletedCallback": "https://example.org/deleted_archives",
"description": "Archived dataset with experiment data from 2021."
}Responses
The request has been accepted and the result is the archive Id. Also URI of the Archive in form https://$PROVIDER_HOST/api/v3/oneprovider/archives/{id} is returned in the response Location header.
Archive Id.
Example
{
"archiveId": "ae6f78c89a97c9e78e891105f703bcb8"
}Invalid request.
error Object describing an error.
Object describing an error.
String identifying the error type. Does not change between error instances.
Human readable error description. May contain information specific to given error instance.
Details about the error instance. The object schema is specific to each error type.
ctx required Technical information about error.
Technical information about error.
Timestamp in milliseconds.
Release version of a product, for example "25.1".
Identifier of a service instance — either Oneprovider Id or "onezone".
Domain of the service.
Build version of the service.
Short name of service.
Revision of onedata-errors repo, where the error is defined.
Module where the error occurred.
Line number in the module where the error occurred.
Example
{
"error": {
"id": "example",
"description": "This is an example error.",
"ctx": {
"timestamp": 1750762637248,
"serviceReleaseVersion": "25.1",
"serviceId": "a7ee2722915b8bf7fb064eb65087cc61ch9357",
"serviceDomain": "dev-oneprovider-krakow.default.svc.cluster.local",
"serviceBuildVersion": "231-g8d26f953c3",
"service": "opw",
"onedataErrorsRevision": "ca3af8cb",
"module": "metadata_req",
"line": 45
}
}
}Forbidden request.
error Object describing an error.
Object describing an error.
String identifying the error type. Does not change between error instances.
Human readable error description. May contain information specific to given error instance.
Details about the error instance. The object schema is specific to each error type.
ctx required Technical information about error.
Technical information about error.
Timestamp in milliseconds.
Release version of a product, for example "25.1".
Identifier of a service instance — either Oneprovider Id or "onezone".
Domain of the service.
Build version of the service.
Short name of service.
Revision of onedata-errors repo, where the error is defined.
Module where the error occurred.
Line number in the module where the error occurred.
Example
{
"error": {
"id": "example",
"description": "This is an example error.",
"ctx": {
"timestamp": 1750762637248,
"serviceReleaseVersion": "25.1",
"serviceId": "a7ee2722915b8bf7fb064eb65087cc61ch9357",
"serviceDomain": "dev-oneprovider-krakow.default.svc.cluster.local",
"serviceBuildVersion": "231-g8d26f953c3",
"service": "opw",
"onedataErrorsRevision": "ca3af8cb",
"module": "metadata_req",
"line": 45
}
}
}Dataset not found.
error Object describing an error.
Object describing an error.
String identifying the error type. Does not change between error instances.
Human readable error description. May contain information specific to given error instance.
Details about the error instance. The object schema is specific to each error type.
ctx required Technical information about error.
Technical information about error.
Timestamp in milliseconds.
Release version of a product, for example "25.1".
Identifier of a service instance — either Oneprovider Id or "onezone".
Domain of the service.
Build version of the service.
Short name of service.
Revision of onedata-errors repo, where the error is defined.
Module where the error occurred.
Line number in the module where the error occurred.
Example
{
"error": {
"id": "example",
"description": "This is an example error.",
"ctx": {
"timestamp": 1750762637248,
"serviceReleaseVersion": "25.1",
"serviceId": "a7ee2722915b8bf7fb064eb65087cc61ch9357",
"serviceDomain": "dev-oneprovider-krakow.default.svc.cluster.local",
"serviceBuildVersion": "231-g8d26f953c3",
"service": "opw",
"onedataErrorsRevision": "ca3af8cb",
"module": "metadata_req",
"line": 45
}
}
}Internal server error.
error Object describing an error.
Object describing an error.
String identifying the error type. Does not change between error instances.
Human readable error description. May contain information specific to given error instance.
Details about the error instance. The object schema is specific to each error type.
ctx required Technical information about error.
Technical information about error.
Timestamp in milliseconds.
Release version of a product, for example "25.1".
Identifier of a service instance — either Oneprovider Id or "onezone".
Domain of the service.
Build version of the service.
Short name of service.
Revision of onedata-errors repo, where the error is defined.
Module where the error occurred.
Line number in the module where the error occurred.
Example
{
"error": {
"id": "example",
"description": "This is an example error.",
"ctx": {
"timestamp": 1750762637248,
"serviceReleaseVersion": "25.1",
"serviceId": "a7ee2722915b8bf7fb064eb65087cc61ch9357",
"serviceDomain": "dev-oneprovider-krakow.default.svc.cluster.local",
"serviceBuildVersion": "231-g8d26f953c3",
"service": "opw",
"onedataErrorsRevision": "ca3af8cb",
"module": "metadata_req",
"line": 45
}
}
}