Important
Help us improve Directus Sync by sharing your feedback! Take a quick survey about your usage here: https://forms.gle/LnaB89uVkZCDqRfGA
The directus-sync
command-line interface (CLI) provides a set of tools for managing and synchronizing the schema and
collections within Directus across different environments.
By leveraging Directus's REST API, it aligns closely with the native actions performed within the application, ensuring a high fidelity of operation.
Updates are granular, focusing on differential data changes rather than blunt table overwrites, which means only the necessary changes are applied, preserving the integrity and history of your data.
Moreover, directus-sync
organizes backups into multiple files, significantly improving readability and making it
easier to track and review changes. This thoughtful separation facilitates a smoother version control process, allowing
for targeted updates and clearer oversight of your Directus configurations.
Table of Contents
- Directus Sync
- Node.js 18 or higher
directus-extension-sync
installed on your Directus instance. See the installation instructions.
The CLI is available using the npx
command.
npx directus-sync <command> [options]
Here's how to use each command in the CLI:
npx directus-sync pull
Retrieves the current schema and collections from Directus and stores them locally. This command does not modify the database.
It also retrieves the specifications (GraphQL & OpenAPI) and stores them locally.
It gets specifications from the /server/specs/*
endpoints:
npx directus-sync diff
Analyzes and describes the difference (diff) between your local schema and collections and the state of the Directus instance. This command is non-destructive and does not apply any changes to the database.
npx directus-sync push
Applies the changes from your local environment to the Directus instance. This command pushes your local schema and collection configurations to Directus, updating the instance to reflect your local state.
Options are merged from the following sources, in order of precedence:
- CLI arguments
- Environment variables
- Configuration file
- Default values
These options can be used with any command to configure the operation of directus-sync
:
-
-c, --config-path <configPath>
Change the path to the config file. Default paths are:./directus-sync.config.js
,./directus-sync.config.cjs
or./directus-sync.config.json
. -
-d, --debug
Display additional logging information. Useful for debugging or verifying whatdirectus-sync
is doing under the hood. -
-u, --directus-url <directusUrl>
Specify the Directus instance URL. Alternatively, set theDIRECTUS_URL
environment variable. -
-t, --directus-token <directusToken>
Provide the Directus access token. Alternatively, set theDIRECTUS_TOKEN
environment variable. If provided, thedirectus-email
anddirectus-password
options are ignored. -
-e, --directus-email <directusEmail>
Provide the Directus email. Alternatively, set theDIRECTUS_ADMIN_EMAIL
environment variable. -
-p, --directus-password <directusPassword>
Provide the Directus password. Alternatively, set theDIRECTUS_ADMIN_PASSWORD
environment variable. -
--dump-path <dumpPath>
Set the base path for the dump. This must be an absolute path. The default is"./directus-config"
. -
--collections-path <collectionPath>
Specify the path for the collections dump, relative to the dump path. The default is"collections"
. -
-o, --only-collections <onlyCollections>
Comma-separated list of directus collections to include duringpull
push
ordiff
process. -
-x, --exclude-collections <excludeCollections>
Comma-separated list of directus collections to exclude duringpull
push
ordiff
. Can be used along withonly-collections
. -
--preserve-ids <preserveIds>
Comma-separated list of directus collections to preserve the original ids during thepull
orpush
process.
Possible collections are:dashboards
,operations
,panels
,roles
andtranslations
.
flows
andfolders
ids are always preserved.
The value can be*
orall
to preserve ids of all collections, when applicable. -
--snapshot-path <snapshotPath>
Specify the path for the schema snapshot dump, relative to the dump path. The default is"snapshot"
. -
--no-snapshot
Do not pull and push the Directus schema. By default, the schema is pulled and pushed. -
--no-split
Indicates whether the schema snapshot should be split into multiple files. By default, snapshots are split. -
--specs-path <specsPath>
Specify the path for the specifications dump (GraphQL & OpenAPI), relative to the dump path. The default is"specs"
. -
--no-specs
Do not dump the specifications (GraphQL & OpenAPI). By default, specifications are dumped. -
-f, --force
Force the diff of schema, even if the Directus version is different. The default isfalse
. -
-h, --help
Display help information for thedirectus-sync
commands.
The directus-sync
CLI also supports a configuration file. This file is optional. If it is not provided, the CLI will
use the default values for the options.
The default paths for the configuration file are ./directus-sync.config.js
, ./directus-sync.config.cjs
or ./directus-sync.config.json
. You can change this path using the
--config-path
option.
The configuration file can extend another configuration file using the extends
property.
This is an example of a configuration file:
// ./directus-sync.config.js
module.exports = {
extends: ['./directus-sync.config.base.js'],
debug: true,
directusUrl: 'https://directus.example.com',
directusToken: 'my-directus-token',
directusEmail: 'admin@example.com', // ignored if directusToken is provided
directusPassword: 'my-directus-password', // ignored if directusToken is provided
directusConfig: {
clientOptions: {}, // see https://docs.directus.io/guides/sdk/getting-started.html#polyfilling
restConfig: {}, // see https://docs.directus.io/packages/@directus/sdk/rest/interfaces/RestConfig.html
}
dumpPath: './directus-config',
collectionsPath: 'collections',
onlyCollections: ['roles', 'permissions', 'settings'],
excludeCollections: ['settings'],
preserveIds: ['roles', 'panels'], // can be '*' or 'all' to preserve all ids, or an array of collections
snapshotPath: 'snapshot',
snapshot: true,
split: true,
specsPath: 'specs',
specs: true,
};
In addition to the CLI commands, directus-sync
also supports hooks. Hooks are JavaScript functions that are executed
at specific points during the synchronization process. They can be used to transform the data coming from Directus or
going to Directus.
Hooks are defined in the configuration file using the hooks
property. Under this property, you can define the
collection name and the hook function to be executed.
Available collection names
are: dashboards
, flows
, folders
, operations
, panels
, permissions
, presets
, roles
, settings
and translations
.
For each collection, available hook functions are: onQuery
, onLoad
, onSave
, and onDump
.
These can be asynchronous functions.
During the pull
command:
onQuery
is executed just before the query is sent to Directus for get elements. It receives the query object as parameter and must return the query object. The second parameter is the Directus client.onDump
is executed just after the data is retrieved from Directus and before it is saved to the dump files. The data is the raw data received from Directus. The second parameter is the Directus client. It must return the data to be saved to the dump files.onSave
is executed just before the cleaned data is saved to the dump files. The "cleaned" data is the data without the columns that are ignored bydirectus-sync
(such asuser_updated
) and with the relations replaced by the SyncIDs. The first parameter is the cleaned data and the second parameter is the Directus client. It must return the data to be saved to the dump files.
During the push
command:
onLoad
is executed just after the data is loaded from the dump files. The data is the cleaned data, as described above. The first parameter is the data coming from the JSON file and the second parameter is the Directus client. It must return the data.
Here is an example of a configuration file with hooks:
// ./directus-sync.config.js
module.exports = {
hooks: {
flows: {
onDump: (flows) => {
return flows.map((flow) => {
flow.name = `🧊 ${flow.name}`;
return flow;
});
},
onSave: (flows) => {
return flows.map((flow) => {
flow.name = `🔥 ${flow.name}`;
return flow;
});
},
onLoad: (flows) => {
return flows.map((flow) => {
flow.name = flow.name.replace('🔥 ', '');
return flow;
});
},
},
},
};
Warning
The dump hook is called after the mapping of the SyncIDs. This means that the data received by the hook is already
tracked. If you filter out some elements, they will be deleted during the push
command.
You can use onQuery
hook to filter out elements. This hook is executed just before the query is sent to Directus,
during the pull
command.
In the example below, the flows and operations whose name starts with Test:
are filtered out and will not be tracked.
// ./directus-sync.config.js
const testPrefix = 'Test:';
module.exports = {
hooks: {
flows: {
onQuery: (query, client) => {
query.filter = {
...query.filter,
name: { _nstarts_with: testPrefix },
};
return query;
},
},
operations: {
onQuery: (query, client) => {
query.filter = {
...query.filter,
flow: { name: { _nstarts_with: testPrefix } },
};
return query;
},
},
},
};
Warning
Directus-Sync may alter the query after this hook. For example, for roles
, the query excludes the current admin
role.
The example below shows how to disable the flows whose name starts with Test:
and add the flow name to the operation.
const { readFlow } = require('@directus/sdk');
const testPrefix = 'Test:';
module.exports = {
hooks: {
flows: {
onDump: (flows) => {
return flows.map((flow) => {
flow.status = flow.name.startsWith(testPrefix)
? 'inactive'
: 'active';
});
},
},
operations: {
onDump: async (operations, client) => {
for (const operation of operations) {
const flow = await client.request(readFlow(operation.flow));
if (flow) {
operation.name = `${flow.name}: ${operation.name}`;
}
}
return operations;
},
},
},
};
Like the collections hooks, the snapshot hooks are defined in the configuration file using the hooks.snapshot
property. Under
this property, you can define the hook functions to be executed.
Available hook functions are: onLoad
, onSave
:
onLoad
is executed during thepush
anddiff
processes, just after the data is loaded from the files, and before it is sent to Directus.onSave
is executed during thepull
process, just before the data is saved to the files.
Note
This function can be asynchronous. It receives the snapshot object and the Directus client as parameters and must return the snapshot object.
Here is an example of a configuration file that exclude some fields when loading the snapshot. This will be similar for
the onSave
hook.
// ./directus-sync.config.js
module.exports = {
hooks: {
snapshot: {
/**
* @param {Snapshot} snapshot
* @param {DirectusClient} client
*/
onLoad: async (snapshot, client) => {
// Remove some fields from the snapshot
const fieldsToExclude = {
my_model: ['date_created', 'user_created'],
};
const collections = Object.keys(fieldsToExclude);
const nodeFilter = (node) => {
const { collection } = node;
return !(collections.includes(collection) && fieldsToExclude[collection].includes(node.field));
}
snapshot.fields = snapshot.fields.filter(nodeFilter);
snapshot.relations = snapshot.relations.filter(nodeFilter);
return snapshot;
},
},
},
};
Note
For more information about the snapshot object, see the Snapshot interface.
npx directus-sync helpers untrack --collection <collection> --id <id>
Removes tracking from an element within Directus. You must specify the collection and the ID of the element you wish to stop tracking.
Permissions should be unique regarding the role
, collection
, and action
.
Unfortunately, Directus does not enforce this uniqueness.
This can lead to unexpected behavior, such as missing ids or other permissions fields.
More details can be found in the Directus issue #21965.
If you have permission duplicates, you can use the following command to remove them.
npx directus-sync helpers remove-permission-duplicates --keep <keep>
--keep <keep>
: The permission's position to keep,first
orlast
. The default islast
.
This command will keep the last
(or first
) permission found and remove the others for each duplicated
group role
, collection
, and action
.
flowchart
subgraph Pull[Get elements - for each collection]
direction TB
B[Create query for all elements]
-->|onQuery hook|C[Add collection-specific filters]
--> D[Get elements from Directus]
--> E[Get or create SyncId for each element. Start tracking]
--> F[Remove original Id of each element]
-->|onDump hook|G[Keep elements in memory]
end
subgraph Post[Link elements - for each collection]
direction TB
H[Get all elements from memory]
--> I[Replace relations ids by SyncIds]
--> J[Remove ignore fields]
--> K[Sort elements]
-->|onSave hook|L[Save to JSON file]
end
A[Pull command] --> Pull --> Post --> Z[End]
Coming soon
Coming soon
directus-sync
tracks the following Directus collections:
- dashboards
- flows
- folders
- operations
- panels
- permissions
- presets
- roles
- settings
- translations
For these collections, data changes are committed to the code, allowing for replication on other Directus instances. A
mapping table links Directus instance IDs with SyncIDs, managed by the directus-extension-sync
.
Roles are tracked, but the main administrator role is not tracked. This is because the administrator role is a system role and is automatically created by Directus. It will cause conflicts if it is tracked.
The main administrator is the role of the curent user, used to authenticate the directus-sync
commands.
Global and role based presets are tracked (even the administrator role based presets). However, the users' presets are not tracked. This is because the users are not tracked and any relation with the users will cause conflicts.
To utilize the directus-sync
tool, it is imperative to install the directus-extension-sync
extension on your
Directus instance. This extension acts as a bridge between directus-sync
and Directus, managing the crucial mapping
table that correlates the SyncIDs with Directus's internal IDs.
The directus-extension-sync
must be added to each Directus instance involved in the synchronization process, whether
as a source or a destination. Follow the installation instructions provided in
the directus-extension-sync
repository
to add this extension to your Directus setup.
directus-sync
operates on a tagging system similar to Terraform, where each trackable element within Directus is
assigned a unique synchronization identifier (SyncID). This system is key to enabling version control for the
configurations and schema within Directus. Here is a step-by-step explanation of how directus-sync
functions:
Upon execution of the pull
command, directus-sync
will:
- Scan the specified Directus collections, which include dashboards, flows, folders, operations, panels, permissions, presets, roles, settings and translations.
- Assign a SyncID to each element within these collections if it doesn't already have one.
- Commit the data of these collections into code, allowing for versioning and tracking of configuration changes.
This SyncID tagging facilitates the replication of configurations across different instances of Directus while maintaining the integrity and links between different entities.
Note
The original IDs of the flows are preserved to maintain the URLs of the webhook
type flows.
The original IDs of the folders are preserved to maintain the associations with fields of the file
and image
types.
Tip
You can use the --preserve-ids
option to preserve the original ids of some collections.
Eligible collections are collections using UUID: dashboards
, operations
, panels
, roles
and translations
.
If you have already used the pull
command, you may use the untrack
helper to remove the id tracking of an element
before using this option.
Since it's not possible to add tags directly to entities within Directus, directus-sync
uses a mapping table that
correlates the SyncIDs with the internal IDs used by Directus. This mapping is essential for the synchronization
process, as it ensures that each element can be accurately identified and updated across different environments.
The synchronization process is split into two main commands:
-
diff
: This command performs a comparison between the local JSON files generated bypull
and the current state of a Directus instance. It outlines the elements that need to be created, updated, or deleted to achieve synchronization. -
push
: This command executes the actual synchronization plan, applying the necessary changes to the Directus instance. It handles dependencies and circular dependencies carefully by potentially running the synchronization process multiple times until the Directus instance is fully in sync with the JSON definitions.
The Directus schema, which defines the data modeling and user interface, is managed by the Directus API. However, for
better code repository management, directus-sync
stores the schema elements in separate files organized within a clear
directory structure. This separation allows developers to easily track changes to the schema and apply version control
principles to the database structure.
Elements that are not meant to be tracked, such as user activities and logs, are not affected by the synchronization process. Certain fields are specifically ignored during synchronization because they are not relevant for version control purposes, such as creation dates and the identity of the user who created an entity.
The strength of directus-sync
lies in its ability to maintain consistent and reproducible configurations across
multiple environments. It ensures that only the necessary changes are made, avoiding unnecessary recreation of
configurations and maintaining the relationships between tracked and non-tracked entities. This selective updating is
what makes directus-sync
a robust tool for managing Directus instances in a team or multi-environment setup.
By following these mechanisms, directus-sync
streamlines the development workflow, allowing for local changes to be
efficiently deployed to various environments, all while keeping the Directus instances synchronized and
version-controlled.
When upgrading Directus, it is important to update the configurations pulled by directus-sync
. Here is a general guide
to upgrading Directus using directus-sync
:
- Run Directus on Version A: Start with your current Directus setup that is actively running on version A.
- Push Configuration (if needed): Use
npx directus-sync push
to push the latest configurations to your Directus instance. This step ensures that all your current settings are up-to-date on the server. - Stop the Directus Server: Shut down your Directus server to prepare for the upgrade.
- Update to Version B: Change the Directus version to B in your configuration files or update scripts (
e.g.,
package.json
,docker-compose.yml
orDockerfile
). - Restart and Migrate: Start the Directus server to initiate the upgrade. Directus will automatically run migration scripts necessary to update the database and apply new system configurations.
- Pull New Configuration: Once Directus is stable on version B, execute
npx directus-sync pull
to download the latest configuration snapshot. This action captures any changes or migrations that occurred during the upgrade from version A to B.