DryvIQ Platform Release Notes
Version: 5.7.2
Release Date: June 7, 2024
DryvIQ Platform version 5.7.1 2 was available on May 17June 7, 2024. This release introduces customized Discover scan setup and dashboards as well as other exciting new features and enhancementsis a maintenance release that focuses on enhancements and bug fixes. Learn more about the release below. For detailed information about individual items, refer to the DryvIQ Platform Documentation Center.the DryvIQ Platform Documentation Center.
New Features and Enhancements
Discover
When the Metadata entity type is being used as part of Discover, the columns in the Discover Results now dynamically update to display the metadata received from the connected platforms.
Migrate
The Jobs export now includes the last failure message and status for each job on the export.
Items flagged for retry that no longer exist will now be marked as “Ignored,” and an audit log message will be added for the item.
Improvements were made to Simulation Jobs to prevent jobs failing with an "Object reference" error when targeting a destination folder that doesn't exist.
...
Fixes
Govern
The Microsoft Office 365 connector was updated to fix an issue with connecting to a site using a tenant level connection and running a policy that requires access rules.
Updates were made to prevent reporting duplicate items in the policy scan results for PostgreSQL databases when data sources are assigned to more than one policy.
Migrate
The SharePoint connectors were updated to correct an issue that would sometimes correlate an old version of an item as the current version.
Encoded characters can now be used the destination path when creating a SharePoint connector in DryvIQ.
When paging sites at the Microsoft Office 365 tenant level, all sites for a tenant will be available in the Locations picker.
The Dropbox for Business and Dropbox for Business Teams Folders connectors were updated to produce better error messages when attempting to impersonate an invalid or unknown Dropbox user account during author preservation.
The HCP Anywhere connector was updated to prevent "The application was unable to obtain a valid access token to the storage platform" errors in jobs when HCP Anywhere source system is under load.
The NFS connector was updated to ensure Execute access rights were properly applied to NFS items.
Updates were made to prevent 500 errors and rate limit exceptions from disabling a connector.
DryvIQ will return null to prevent an error for files with unparsable created on, modified on, or last accessed time.
Link remediation was updated to remediate links to files outside the current job.
Improved error messaging was added to identify when there is an issue with the user or group map configuration.
When DryvIQ encounters a write/delete collision during a copy job (one-way transfer), it will prefer the write and treat it as a new item, generating a new platform ID.
Settings and Miscellaneous
Updates were made to improve loading of the DryvIQ API documentation.
DryvIQ deployments in Kubernetes environments now respect log level set on the Performance page in the Settings.
| Info |
|---|
Updating the DryvIQ Platform does not update the Command-line Interface (CLI). If you use the CLI, you should update it when you update to a new version of DryvIQ to ensure the CLI is current. Run the CLI install command to update the CLI. |
...
Previous Release Notes
| Expand |
|---|
| title | 5.7.1 (May 17, 2024) |
|---|
|
Insights now includes a new Managed Custom Insights option. Custom Insights allow you to create your own custom dashboards to report on the information most relevant to your data discovery needs. Learn more here. Discover scan setup is fully available through the user interface and includes the ability to customize everything from the data sources and discover categories that should be included in the scan to the schedule the scan will use. Learn more here. You can now save a view as a document set and use the saved document set as the data source for a new Discover scan. Learn more here. The Locations page was renamed Scans to reflect that this page lists the Discover scans that have been created. The Refine feature on the Scans page (formally Locations) has two new options: Has child scans? and Is a Document set? so you can quickly filter the list for scan for mapped locations and document sets. Learn more here. The “Scan All” options has been removed to prevent all scans from being run accidentally. You can still select and run multiple scans. Improvements were made to how you filter the Results by location. This option was removed from the filter panel and was added to the new Select scans option on the top of the Results page. The new selection modal simplifies how you select the scans you view by allowing you to search for a specific scan, view all scans, or browse through the 50 most recent scans. Learn more here. The Results export now defaults to a maximum of 100,000 items to ensure performance. The export will truncate anything selected over this value when generating the export. The maximum value can be adjusted using the catalog:query_max_export value in the configuration settings if a different default value is preferred. Learn more here. The date fields in the Results export now use ISO 8601 formatting. Learn more here.
The Self-Service Migration Management charts were updated so “Processed” is now “Revised in Transfer” to match the charts on the Overview page of the individual job reports. The export for jobs that failed to start now includes the last failure message and status for the jobs.
The SharePoint connector was updated to correct an issue that would sometimes correlate an old version as the current when reading an item. Items that were marked for Retry but no longer exist will now be marked as Ignored with an audit log message. Updates were made to prevent 500 errors and rate limit exceptions from disabling connections. When a write event occurs on the source and a move out of job scope event on the destination is detected, the moved file will be deleted. All sites for a tenant will be available when paging through the Locations picker at the Office 365 tenant level. Execute access rights are properly applied to NFS items. Link remediation for files outside the job now properly remediates. DryvIQ will return null to prevent an error for files with an unparsable created on, modified on, or last accessed date.
|
...
| Info |
|---|
Updating the DryvIQ Platform does not update the Command-line Interface (CLI). If you use the CLI, you should update it when you update to a new version of DryvIQ to ensure the CLI is current. Run the CLI install command to update the CLI. |
...
| Expand |
|---|
| title | 5.6.3 (April 4, 2024 |
|---|
|
| Info |
|---|
The new File Versions entity type must be added to your Discover scan in order for this information to be available on the Results page. |
A new File Versions entity type is available that provides file version details such as the number of versions for a file and the the total amount of bytes for all versions. In addition to being used as part of the Discover Results to provide version insights, this entity type can be used as part of a Govern policy or you can upload individual files against the entity type if you prefer.
Updates were made to the Validation page to prevent “infinity%” from displaying. A fix was made to correct an issue with the URL endpoint not providing connection data to the URL resolver. Items with more than 200 versions will be transferred. To prevent audit trail issues, the SharePoint Online connectors were updated to delete folder content individually when the retention policy is applied. The SharePoint Online connectors were updated to fix an issue that prevents DryvIQ from creating libraries that have the same name as the site name. The Box connector was updated to correct issues with Box Notes rendering due to list items and headings that used advanced formatting. An update was made to prevent Google native docs from showing with 0 bytes during simulation mode when versioning is enabled. Note that only the size of the current version can be calculated from the API during simulation.
|
| Expand |
|---|
| title | 5.6.2 (March 7, 2024) |
|---|
|
Additional options have been added on the in the Locations page that allow you to view results and insights for each location. Learn more. All location pickers have been updated to display the connection user or impersonated user under the path.
A new job setting can be configured for delta job runs to skip destination events in scenarios where the subsequent job run occurs after an initial run into an empty destination. This setting can only be set using the API and must be set per job run. Learn more here. Records in the Transfer Item audit will now be purged after 90 days. The default value can be updated in the configuration options. The Items job report page now allows you to display Mime type as one of the column headers. It displays the mime type for five types of Google documents: Google Jamboard, Google Sites, Google Maps, Google Forms, and Lucidchart. Learn more here. Items reported with an unknown status are now reported on the Overview and Items reports. The roll up reports now include a User Mappings page like the individual job report. This allows you to review all the user mappings for the jobs included in the report instead of having to go to the individual job to review the user mapping results. Updates were made to improve the performance of the Links export when an large number of links are included in the exported report.
When you cancel the location picker modal for a Dropbox connection when creating a job, DryvIQ now properly sends Dropbox a cancelation token to cancel any additional paging requests. The Dropbox connector was updated to request and use refresh tokens for authentication. Folders with permissions set to Deny now show up as Flagged. Graphs were updated to show 0 instead of negative numbers. An error no longer occurs when duplicating jobs in the user interface without editing the root paths. An update was made to ensure the the metadata filter reason is populated in database when an item is excluded by an “Include” policy. Sorting by job duration now displays in proper chronological order. A null check was added to the Google connector to ensure links located on the root of the Google connection can be detected and resolved. The Job column on the Items page now sorts properly.
|
| Expand |
|---|
| title | 5.6.1 (February 9, 2024) |
|---|
|
|
| Expand |
|---|
| title | 5.6 (January 25, 2024) |
|---|
|
Discover now supports adding a location as a mapped location to make adding locations to the Discover scan easier. When you execute the mapping against the location, DryvIQ creates a child locations for every folder one level under the primary location. You can then run the Discover scan at the child location. Learn more here. The DryvIQ Discover results now include duplicate detection. Learn more here.
A new Self-Service Migration Management feature was added so migration administrators can monitor each migration wave separately and easily manage the jobs for the migration waves as needed. Learn more here. Link remediation has been added to the Link Detection feature, so DryvIQ can now update the identified links, eliminating the need for manual remediation. Learn more here. An update was made to the “Move” transfer type to support moving duplicate files (multiple files on the source with the same name and extension) to the destination if the platform supports duplicate names. For SharePoint Online migrations, desktop.ini and .ds_store files will be revised and included in the revised file count.
The Amazon S3 connector now supports permissions. Learn more here. The SMB Network File System connector now has a Connection Mode setting that allows you to create the connection with read-only access. Learn more here. DryvIQ now offers a Hitachi HCP Anywhere connector. Learn more here.
|
| Expand |
|---|
| title | 5.5.6.4043 (December 7, 2023) |
|---|
|
DryvIQ tested and verified the Dropbox connectors (Dropbox for Business and Dropbox for Business Teams Folders) are compatible with the new Dropbox for Business Teams Spaces 2.0. You do not need to update existing Dropbox connections as DryvIQ works with both versions of Dropbox for Business Teams Spaces. The Google Workspace connector was updated with a new new setting that allows DryvIQ to impersonate a file owner when deleting an item. This prevents items from being flagged when the file owner is different than the current impersonated user, requiring less manual remediation. Learn more here.
Because Google does not allow downloading of native link files that appear on the Google platforms, DryvIQ now filters these files out of the migration job and logs the skipped reason for reference. A change was made to the SharePoint Online connectors to refresh the server time zone to ensure proper timestamps for on-premises and customer-hosted deployments.
Restoring deleted items to the destination no longer results in multiple copies being created with # appended to the name. A fix was made to ensure the Reports page in the DryvIQ Cloud environment loads. Filtered items are now properly labeled as “ignored” rather than being added to the flagged items.
|
| Expand |
|---|
| title | 5.5.5 (November 2, 2023) |
|---|
|
A new option was added to the Author Preservation settings that allows you to preserve authorship of the destination root folder. Learn more here. The Box connector was updated to improve how DryvIQ pages large amounts of content to improve transfer performance.
Changes were made to how DryvIQ handles sharing to external users for Microsoft SharePoint Online connectors to ensure requests show up in External Access Invitations. An update was made to the Box connector to ensure duplicate files are not created when DryvIQ encounters errors when transferring content.
|
| Expand |
|---|
| title | 5.5.3 (October 13, 2023) |
|---|
|
A new “Account/Group Unmapped” Revision filter has been added to the Items page so the grid can be filtered to show just unmapped files. Learn more here. To ensure performance, the permissions export available on the Sharing Insights page has been limited to one million rows. Learn more here.
The Network File System (NFS) connector was updated to prevent errors from being produced when using read-only accounts for the connection. The SharePoint Online connectors were updated with the following fixes: Microsoft CSOM will be used to ensure external user notifications are sent when this option is enabled for the connection. Permissions added using shared links will be preserved when migrating between SharePoint connections.
The Office 365 application registration has been updated to show DryvIQ instead of “SkySync.” The Egnyte connector was updated to fix the following issues with metadata mapping: Metadata mapping no longer fails. An issue with setting and retrieving date fields in Egnyte metadata templates has been fixed. This included adding support for metadata timestamp values using the yyyymmdd format. Requesting the metadata schema for an Egnyte connection through the REST API no longer produces an error.
Root permissions are now properly transferred from the source to a SharePoint Online document library.
Jobs will now correctly read the source and destination targets when only the platform ID is used to create the job through the REST API. An update was made to the account mapping logic to include the username in the hashing calculation to improve performance for account maps where only the username is available.
|
| Expand |
|---|
| title | 5.5.1 (September 13, 2023) |
|---|
|
DryvIQ now includes an Intelligent Migration feature that allows you to assign items in the Discover Results to an intelligent migration job. This allows you to take action against your Discover Results to ensure your content is migrated exactly where you want it to be moved based on its make up and sensitivity. Learn more here. You now have the option of making a saved view private or public when creating it. Private views are available only on the machine where the view was originally saved. Public views are available to all users, so you can create a saved view and allow others to use it to review the results and take actions against the content as needed. Learn morehere.
Updates were made to SharePoint online connections to properly manage duplicate folder names on move events. OneDrive for Business connections now properly display user folders so the file path can be specified when creating a job. DryvIQ now successfully creates the target folder when the folder name matches the SharePoint Site name. Migration jobs using SharePoint Online connections now handle European local settings datetime formats correctly on Windows Servers. Scheduling transfer or policy jobs now correctly adds them to the intended scheduler, eliminating duplicate entries in the database. The "Downloading content is not supported for file" message is now an error rather than a warning. Items that receive this message will be added to the Flagged Items so you can choose to retry or ignore the items. This change is not retroactive, so items that received this message previous to 5.5.1 will still be flagged with a warning. Items flagged as “Retry” in a job no longer produce a disconnected tracking error.
|
| Expand |
|---|
| title | 5.5 (August 16, 2023) |
|---|
|
A new Stale Data insights page was added so you can identify your outdated data, also referred to as redundant, obsolete, and trivial (ROT) data. Learn more here.
The forced column setup on first access has been removed. The page now defaults to showing all columns and content for the columns. You need to click Edit columns and adjust the display to reflect the preferred display. The column order was changed to display the most important file details first. Now, the Item Name column includes the path, and this column is followed by the File Details elements in a new order: Bytes, Created on, Created by, Modified on, Modified by, Owner, and Last Touched. A new Owner column was added to the display grid to show who is assigned as the owner of the file. For platforms that don't support a separate owner, this column will default to the author (created by). The Results grid now includes four new sharing columns as part of the Permissions entity type:
1. Shared via External Links to identify files shared with users outside of the internal domain,
2. Shared via Internal Links to identify files shared with users in the internal domain,
3. Shared Link User Addresses to identify the email addresses for the users to whom the files is shared, and
4. Shared Link Group Names to show the group names with whom the file has been shared. Bytes formatting was updated to follow a more standard byte format. Instead of reporting the size in just bytes, we now format the size into a corresponding unit (B, KB, MB, etc.). The Sharing and Permissions filter was renamed Sharing and Access Control. As part of this rename, the sections were reorganized and updated to make the filters more effective. Part of the filter updates was the addition of operators that allow you to add more specific filtering for the user and group sharing. Learn more here. The Filters and Search buttons were renamed Filter and Refine (respectively). In addition, highlighting was added to the buttons to make it clear they have been selected. The filter pills for a location only include the contextual path as this is the most important part of the location, and it should be visible. This applies to all places in Discover where locations are displayed. You can now choose to save a view as a new view or an existing view. If no saved views exist, you have to save the view as a new view. If there are existing views saved, you can choose to update a view (save the current view as an existing view) or create a new saved view. Learn more here. DryvIQ now uses natural sorting. Learn more about how natural sorting works here.
The scan status is now available on the Discover Locations page. The Scan status column display when the last scan ran for each location. If a scan is in progress, this field will display a progress bar to indicate the scan progress. If scans are running for multiple locations, each location will display its own progress bar. The Discover Locations page refreshes every minute. Learn more here.
The last scanned date has been added to the top of the Policy reporting page (next to the active toggle). This allows you to see how old the scan results are when reviewing the policy reports. This date does not automatically refresh. The data only updates if you navigate to another page or manually refresh the page. Updates were made so you don't lose your place when editing a policy. Before, when you clicked Apply changes, the current tracking group would close, and the screen would reset to the top of the page. Now, the tracking group remains open, and the screen stays at the same location on the on the page. When editing policies, new logic was added so the Apply changes button is unavailable if no changes have been made to the page. This provides a visual indicator whether edits have taken place and if saving is required.
The scan status is now available on the Govern Data Sources page. The Status column display when the last scan ran for each location. If a scan is in progress, this field will display a progress bar to indicate the scan progress. If scans are running for multiple locations, each location will display its own progress bar. The Govern Data Sources page does not refresh, so a manual refresh is required to view progress. Learn more here.
A new Active User Status entity type was added. It will identify if the owner and/or creator of a file is active and can identify if users with whom the file has been shared are active. The File Details entity type includes two new fields:
1. Owner: This will identify the owner of a file. Some platforms (such as Box and Google) allow an owner who is different from the creator to be assigned to a file. If owner is not supported, the creator will be considered the owner.
2. Last Touched: This is the last time the file was touched. It uses the most recent accessed date or last modified date based on which date the platform supports.
The Licensing page now displays the byte limits set for both Migrate and Govern/Discover. We also optimized how we calculate the volume used, which may cause your volume used to go up slower than it did previously. There will be no retroactive changes to your calculations, but this will affect all jobs run after you upgrade to the 23.08.16162131 release. Please see the License page for additional information on how usage will be calculated going forward.
For jobs using Google Drive as the source connection, a permission reset will now properly reapply permissions to converted Google Docs. Updates were made to ensure newly uploaded items on the source are transferred on delta runs. If a job that is part of a connection pool contains an invalid URL, the job will not run and will be flagged as “Failed to start.” This message will be listed in the log, and the failed job will not impact the other jobs associated with the connector. Upgrade scripts no longer fail due to a NULL extension. Updates were made to the Box connector to correct an issue where only 100 permissions (collaborations in Box) were being returned.
|
| Expand |
|---|
| title | 5.4.5 (August 3, 2023) |
|---|
|
When a permissions reset is performed after an initial job execution, newly applied permissions are now correctly recorded in the audit log. If a job exceeds the maximum number of items per container based on the platform limit, the containers will be flagged as “Limit Failures” in the user interface and in the logs. Updates were made to prevent JSON files in private cloud environments from including duplicate and empty keys.
|
| Expand |
|---|
| title | 5.4.4 (June 29, 2023) |
|---|
|
Connector credentials can now be updated to fix a disconnected connection. Samba (Linux NFS): The connector was updated to prevent link detection from failing. Samba (Linux NFS): Updates were made to the connector to improve its handling of locked files so it properly reports them as locked. Update logging was also added as part of this fix to assist with troubleshooting for this connector.
|