Repository Import

Note

Only available in Sonatype Nexus Repository Pro. Interested in a free trial? Start here.

The Repository - Import external files task takes content from a directory on the server and imports it into a repository in Nexus Repository. This allows you to bring external content into a repository from any source.

Planning and Requirements

Before using this task, review the following considerations:

Store content in a location available to any node in highly available (HA) deployments
Any available node is used to run the steps of the import task so the import content needs to be accessible to all nodes in the cluster. This is commonly the same store location for the cluster.
Version Compatibility Requirements
The source and target instances must be the same version when importing between Nexus Repository 3 instances. You may import files from the latest Sonatype Nexus Repository 2 version to upgrade.
Migrated Content
Import is limited to artifacts and will not include any server configurations or metadata such as tags. Metadata and checksums are generated automatically on import and do not maintain exported attributes. The Blob created and Blob updated attributes are set to the time generated during import.
Import does not include hidden files and directories (e.g., those with names that start with a period).
Space Requirements
Copying all components will double the assets and space on disk; hard linking does not do this but will take additional space to store all of the metadata.
Location Requirements
The source directory must be writable by the user running Nexus Repository 3. The import history is stored in the source directory to enable incremental imports.
Timing
This task can take some time to run, and multiple imports are not done in parallel. Only one import/export task will run at a time. Each task needs to be completed before the next one will run. Other import tasks are queued to run after the initial one finishes.
Frequency
You may configure the task to run manually or on a schedule depending on your needs.
Performance
We do not recommend that you import directly to object storage such as S3 as doing so can result in poor performance.
Deployment Policy Impacts for Hosted Repositories
If your hosted repository's deployment policy is set to Disable Redeploy, this will prevent overwriting and updating existing assets during import.
Hard Link Requirements
- To use hard links, the target repository blob store must be on the same file system as the source.
- Hard linking is not supported for object storage blob stores such as S3.

Configuration

Use the following steps to create a new import task.

From the Server Administration panel navigate to Tasks and select the Create task button
Select the task, Repository - Import external files from the task list
Fill out the import configuration from the options below:

Task name
A unique name to reference the task.
Notification email
The email address where notifications on the status of the import task are sent.
Send notification on
A notification may be sent only if the task fails or on either a success or failure.
Target repository
The repository into which to import content. This list will only be populated with Hosted repositories of the supported formats.
Source directory
The directory from which to import content. The files must be in the directory structure that supports the desired format.
Batch Size
Memory optimization feature for troubleshooting performance issues. We recommend leaving this at its default value of 1000. The minimum value is 1.
Enable Hard Links
For some formats, enabling this option will create hard links for the blobs in the source directory to be used in the target directory. A physical copy of the blobs will not be created in the target directory. This can save time if the source directory will remain accessible by the current instance.
Task frequency
Configure the job to run automatically at a specific time or manually as a run-once task. Often import jobs are manually run. Scheduled jobs will only import files when the artifacts are newly added or missed from a previous import.

Ensure you have read the hard link requirements under Planning and Requirements.

Running the Task

Note

Only one import/export task can run at a time. If you attempt to run more than one at a time, the others will be queued to run after the initial one completes.

When the task runs, it generates a .nexus/nxrm-import-export-task/ directory inside of the source directory. This .nexus/nxrm-import-export-task/ directory contains metadata that Nexus Repository uses for future imports so that it will not import content multiple times if the task is run more than once. This will allow you to keep placing content into this directory while only importing newer content if you run the task again.

As Nexus Repository processes the import, it will log every single file it imports to the task log; a new task log is created each time a task runs. If an error occurs with any particular file, Nexus Repository will simply log that fact, and then continue importing the rest of the content.

Resetting Data (.dat) Files

If you need to reset the import data files, run the following command:

rm .nexus/nxrm-import-export-task

Deleting Import Task Source Cache Files Inside Nexus Repository 2 Repository Storage Directory

Should you need to reset an import, you will need to delete the cache.

The following steps provide a workaround for deleting .dat files inside a Nexus Repository 2 storage directory where the .dat files are stored in a .nexus directory instead of .nexus/nxrm-import-export-task/:

Locate the source .nexus directory of the import
1. In Nexus Repository 2, the default location would be sonatype-work/nexus/storage/<repoid>/.nexus
Use a command like the following to find and delete all the import cache files:

find sonatype-work/nexus/storage/releases/.nexus -type f -name 'imported-files*.dat' -delete

Supported Formats

You can import content into repositories of any format listed below. Some formats require that content be structured in a certain manner:

Format	Since	Directory Layout	Hard Link Support Since
Raw	3.23.0	No specific layout is required, content will simply be imported exactly as laid out in the source directory	3.31.0
Maven	3.23.0	Files need to be laid out in the usual maven GAV structure, /(groupId)/(artifactId)/(version)/(artifactId-version.extension) For example: {rootdir}/com/mycompany/awesome-artifact/1.0/awesome-artifact-1.0.jar maven-metadata files at each level (G, A, or V) will also be imported.	3.31.0
npm	3.25.0	No specific layout is required, will simply import all tgz packages contained in the source directory	3.34.0
NuGet	3.25.0	No specific layout is required, will simply import all Nuget packages contained in the source directory	3.35.0
Docker	3.26.0	V1 Image metadata must be stored in the file v1/images/{layerId}/json Image content must be stored in the file v1/images/{layerId}/layer Tags must be stored in the file v1/repositories/{name}/tags/{tag} Image index must be stored in the file v1/repositories/{name}/images/{image} V2 Blobs must be stored in path /v2/-/blobs/{digest} Manifests must be stored in file /v2/{name}/manifests/{tag} Examples: (Note the &#58 in the following path, this is simply an ASCII encoded colon ':' as Windows file systems do not support this character in the filename) {rootdir}/v2/-/blobs/sha256&#5803f4658f8b782e12230c1783426bd3bacce651ce582a4ffb6fbbfa2079428ecb {rootdir}/v2/helloworld/manifests/latest {rootdir}/v2/helloworld/manifests/1.0	3.35.0
PyPI	3.26.0	No specific layout is required, will simply import all archives contained in the source directory (along with any .asc signature files alongside)	3.35.0
RubyGems	3.26.0	No specific layout is required, will simply import all gem files contained in the source directory	3.35.0
Yum	3.26.0	No specific layout is required, will simply import all of the rpm (or drpm) files, as well as any comps.xml files in the source directory	3.35.0