Metadata Mapping
- Andrea Harvey
On This Page
Overview
The metadata import feature allows users to import metadata from platforms that wouldn't otherwise support metadata based on the contents of a delimited text file or a database table. This process can be helpful when importing data produced during an export routine, such as exporting content from a platform not yet supported by the application. The metadata import runs after all content has been processed. It will iterate over each row from the import source and attempt to determine the impacted file along with the desired metadata values. If the file can be determined, the necessary metadata will be added to the impacted file.
Click Edit and select the path on the source platform where the metadata map exists. You can also manually enter the path. Click Use this file to select the file.
Configuring a custom metadata content filter is not applicable when using the metadata import function. DryvIQ will ignore metadata filters.
Verify Platform Metadata Support
You must define your metadata template or the corresponding columns for each platform. For instructions on this setup, please refer to the platform's documentation.
File Format
The import source can be any delimited file or even a database table. The import source needs to contain two types of information for each row in the file: the file path and metadata values.
If you receive an error when importing a CSV file, the file does not meet the expected format. Compare your file to the sample file provided, and edit it before attempting to import it again.
File Path
The first column in the import source must be the file path. However, you can split the path into two columns, one for the directory name and one for the file name.
Valid directory column names: FolderName, Folder_Name, and Folder_Structure
Valid file column names: FileName, File_Name
You can specify the path in one column representing the full path.
Valid column names: FilePath, Path, FullPath, File_Path, and Full_Path
The file path can either be relative to the job source or an absolute path within the source platform. For example, if the job source is /C/files/users/ and the file in question is /C/files/users/jdoe/locations.xls, then the file path in the import source could be /jdoe/locations.xls or /C/files/users/jdoe/locations.xls.
Example
Download a template using the following link: Metadata Import Sample CSV. Notice that any empty fields in the CSV file are empty quotes. Currently, the CSV parser does not allow empty fields to be unquoted.
path,location,order_num,order_date
/C/files/users/jdoe/locations1.xls,US,123456,10/20/2015
/C/files/users/jdoe/locations2.xls,EU,789103,10/21/2015Metadata
The schema (or template) is the metadata schema to use when mapping the metadata (optional). Every other column in the import source is considered a metadata value. To include metadata from multiple schemas, you can structure the column name in the following format "property;schema," where "property" represents the metadata property name and "schema" represents the secondary metadata schema ID.
Ensure the template key/id is used. A template display name will result in a failed import.
Example
Download a template using the following link: Import Metadata from Template and add Custom Metadata Sample CSV. Notice that any empty fields in the CSV file are empty quotes. Currently, the CSV parser does not allow empty fields to be unquoted.
path,employeeid;employeeFile,firstname;employeeFile,lastname;employeeFile,FolderType;hrEmployee
/Employee Files/User/offerletter.docx,13579,John,Doe,Employee LettersHash
If the import file contains a column named "SHA1" or "Hash," the system will validate that the hash in the import file matches the hash in the destination file. DryvIQ will log a warning message if the hash does not match.
Metadata Post Process
The Metadata Import is a post-process activity. Items will transfer to the destination even when encountering an error on the metadata import, as metadata import failures are logged as warnings. Users should review the Log tab for the metadata import job and sort by 'Warnings' to view metadata that was not applied. On subsequent job executions, the failed metadata import items will be marked “Flagged” for remediation, which further helps identify items that may require intervention. Refer to Metadata Import Errors for a list of common import errors and how to correct the issues.
Be sure to let the job run and complete successfully before you check the results.
Reviewing Results
In your source directory where the metadata import file path was configured, review the following:
{{import file name}}-xx-export.csv file indicates metadata import rows that were successful.
{{import file name}}-xx-processed.csv file indicates the rows within the import file that DryvIQ has processed.
{{import file name}}-xx-failures.csv file indicates the metadata import rows that failed to be applied.
Review the destination to ensure metadata values were applied to your content
Rows in the metadata import file that failed to be applied will be returned to the original file so they can be retried on subsequent runs, if needed.
Failed Metadata Import
Metadata that fails to be applied during transfer to the destination will result in an audit warning in the log: “Metadata import failed. The mapped source does not exist or was not transferred successfully to the destination (path=/folder/folder...).”
Failed rows will return to the original import file. Review each item and fix the invalid data in the import file. Refer to Metadata Import Errors for more information about errors and how to correct the file.