DataCore SANsymphony Software-defined Storage Platform

DataCore SANsymphony Software-defined Storage Platform solves difficult storage-related challenges introduced by server and desktop virtualization, cloud computing and more general expansion, business continuity, and disaster recovery initiatives. It creates an active, transparent virtualization layer across disk storage devices to maximize the availability, performance and utilization of datacenters large and small. The integrated set of centrally-managed data protection , provisioning, caching, replication, and migration functions operates uniformly over different models and brands, assimilating current and future equipment non-disruptively. You will find that DataCore SANsymphony cost-effectively speeds up applications, delivers uninterrupted data access and extends the life of your tiered storage investments, while giving you peace of mind.

This topic highlights basic features of the software. Refer to the referenced topics in each section for details. Also see Prerequisites.

Server Groups

Servers in the same group will exchange configuration information and the same operations can be performed from all servers. Server groups can consist of up to 64 DataCore Servers. See Establishing Server Groups.

Multiple server groups can also be managed from the same DataCore Management Console by allowing connections to servers in remote server groups. Multiple connections made from the same console can be easily managed and recalled by saving connection information. See Connecting to a Server Group.

One server group can "partner" with another server group when replicating virtual disks. See Configuring Server Groups for Replication

Features

  • Configuration files for the server group can be backed up to a file and restored if necessary. See Backing Up/Restoring Configuration Files.
  • Support is provided for UPSs attached to DataCore Servers and recognized by the Windows operating system. Write caching is automatically disabled when the DataCore Server is running on UPS battery backup. When power is restored write caching is enabled again. See UPS Support.
  • DataCore SANsymphony may also be installed on hosts to bring advanced SAN features closer to running applications. Hosts can virtualize local storage resources to take advantage of the same performance enhancing features used by DataCore Servers; such as auto-tiering and caching. Hosts can also leverage the performance of locally-attached SSD devices and avoid network latency issues. Installation and configuration on a host server are similar, but there are a few differences. See Hyperconverged Deployments.
  • The Witness feature provides an additional safeguard which pro-actively controls the behavior of mirror synchronization when multi-copy virtual disks lose all mirror paths and management paths and DataCore Servers are unable to communicate. This is a server group setting. See Witness Feature.

Server Maintenance

The Maintenance Mode feature provides a way to move or evacuate virtual disk storage sources created from shared pools from one server to another in order to perform maintenance on a server without loss of high availability. After maintenance is performed, the virtual disk storage sources can be moved back or redistributed among other servers that share the same shared pool. See Maintenance Mode.

Host Groups

Hosts can be grouped to allow operations to be performed simultaneously on the entire group. See Host Groups.

Features

  • Quality of Service (QoS) is the management of I/O traffic in order to meet the demands of different applications or to limit utilization by restricting the amount of I/O operations and data transferred to host groups. QoS settings include limits for the number of I/O operations and the amount of data transferred. See Quality of Service.
  • The Charge Back feature allows IT organizations to track and measure I/O statistics per host group. This feature allows I/O resources to be "metered" so that organizations can determine the resources used per host group and charge back the costs of storage-related expenses. Charge back can also be used to determine usage in order to evaluate future storage needs and appropriate resources more effectively within an organization. See Charge Back.
    • Data usage reports can be created to collect and export I/O statistics for host groups during a specified date range. The report data can be exported in several formats including CSV and XLS.  This data can be imported into billing systems in order to charge back users. See Reports for more information.
    • A Windows PowerShell script file named Export-DcsDataUsageReport.ps1 is included in the DataCore SANsymphony installation folder (default path C:\Program Files\DataCore\SANsymphony) to facilitate exporting data usage statistics for host groups in order to charge back the costs of storage-related expenses to customers. See Exporting Data Usage Reports Script.

Users

The Access Control feature includes role and ownership based authorization which provides refined user controls for the software. Access control allows administrators to fine-tune control and distribute management responsibilities among registered users based on the assignment of roles and virtual disk ownership. See Access Control.

Disk Pools

Disk pools are where physical storage devices—from JBOD enclosures to intelligent storage arrays—are centrally pooled and managed. Disk pools are used to create virtual disks—storage resources that look and behave like a physical disk volume.

When a pool nears depletion, additional physical disks can be transparently added to the pool without disrupting applications. Common maintenance tasks can be performed during normal business hours and with minimum administrator intervention and no downtime.

Any number of disk pools can be created, so storage devices with similar speeds can be grouped. Pools with faster devices can be used for higher priority storage resources.

Features

  • Space can be reserved in the pool for exclusive use by specified virtual disks.
  • Physical disks in pools can be mirrored for fault tolerance.
  • The Automated Storage Tiering feature identifies access patterns in data in the pool and either promotes the data to a higher priority or demotes the data to a lower priority tier based on usage and the storage profile assigned to a virtual disk. The default behavior is to migrate data based on the frequency of read operation only. The Write-aware auto-tiering setting in a storage profile can be used to also migrate data based on the frequency of write operations. See Automated Storage Tiering and Storage Profiles.
  • Enhanced monitoring of available pool space and I/O latency is provided. When thresholds are reached, alerts and email notifications can be provided by configuring an automated task. See System Health Thresholds.
  • The Allocation View tool provides different views of the allocation for disk pools. It displays the allocation and storage temperature per physical disk and also per virtual disk in a pool. See Pool Allocation Tool.
  • The Shared Multi-port Array support feature enables disk pools to be shared by DataCore Servers. See Shared Multi-port Array Support.
  • When a host application deletes large amounts of virtual disk data, virtual disk space will be automatically be reclaimed and returned to the pool without user intervention.
  • Failed or missing physical disks can be purged from a pool in order to make the pool healthy and functional again. During this process, recoverable data will be copied to other physical disks in the same pool and mirrored virtual disks will be resynchronized using a log recovery. See Purging Physical Disks to Repair Failed Pools.
  • DataCore Post-Processing Deduplication can automatically create a deduplication pool that leverages the Data Deduplication feature in Windows Server 2012 R2. See Deduplication.
  • Scripts are provided to automatically disable front-end and mirror access to mirrored virtual disks, causing a failover to another front-end path on a different server when storage performance degradation is detected by a storage latency threshold. When the storage on one server is drastically compromised, temporarily failing over to faster storage on another server will result in improved performance for the virtual disks and host applications until the latency issue can be corrected. See Managing Storage Performance Degradation.
  • DataCore SANsymphony takes advantage of the benefits of 4 KB sector disks by allowing the creation of disk pools from which virtual disks with 4 KB sectors can be created. See 4 KB Sector Support.

For more information, see Disk Pools.

Physical Disks and DataCore disks

Virtual disks that are served to a DataCore Server, via the loopback port or over iSCSI, are automatically discovered on the DataCore Server as a special type of physical disk referred to as a DataCore disk.

  • DataCore disks are better identified in the configuration by appearing under a special classification, DataCore disk, in the DataCore Servers panel for the server to which it is served. They are also listed in the DataCore Server Details page>DataCore disks tab. Details for DataCore disks are provided in the DataCore disk Details page. See DataCore disks.
  • Solid State Disks are better identified in the configuration by a unique icon in the Physical Disk Details page and DataCore Servers panel. See Panel Icons. SSDs are also identified by appending the word "SSD" to the end of the type in the Physical Disk Details page.
  • An I/O latency threshold can be customized for physical disks that are in disk pools and DataCore disks. See System Health Thresholds.

Pass-through Disks

Pass-through disks can be any physical disk that has not been added to a pool, and is simply used as a storage source when virtual disks are created.

  • Pass-through disks can be used temporarily to migrate existing data into the DataCore SANsymphony environment or DataCore SANsymphony can be used as a vehicle to migrate existing data from one environment to another (neither being a DataCore SANsymphony environment).
  • Pass-through disks can also exist permanently in the system and used as storage sources for virtual disks.

For more information, see Pass-through Disks.

Port Connections and Paths

A port is a Fibre Channel or iSCSI port on the DataCore Server or host. A path is a Fibre Channel or iSCSI connection between two ports.

Features

  • DataCore Server ports can be assigned roles to regulate the purpose of the port when paths are created. Limiting the roles that a port supports can regulate the paths and avoid automatic selection of an undesirable port. See Assigning Port Roles and Groups.
  • System Managed Mirroring addresses the complexity of managing multiple mirror paths for numerous virtual disks. Mirror paths can be automatically and silently managed by the software. The creation and management of mirror connections are based on the mirror role assigned to server ports. This is a server group setting. See System Managed Mirroring.
  • The port group feature allows the grouping of DataCore Servers ports in order to create different logical networks which are used when creating front-end or mirror paths. This feature has the ability to ensure that front-end and mirror paths are physically isolated to improve fault tolerance. See Assigning Port Roles and Groups.
  • Redundant mirror are created automatically if redundant connections exist. Hosts are enabled for multipathing by default and when mirrored virtual disks are served to the host, redundant paths are automatically created between the host and both DataCore Servers if possible. Paths can be added or removed as required. See Modifying Paths and Multipathing Support.
  • Hosts can designate preferred DataCore Servers to use when virtual disks are served; paths to the preferred DataCore Servers will be used when possible. See Port Connections and Paths.
  • Preferred paths can be designated for front-end, mirror, and back-end paths. See Modifying Paths.
  • LUNs can be selected in single path mode and starting LUNs can be specified when serving multiple virtual disks at the same time. See Serving Virtual Disks.

For more information, see Port Connections and Paths.

Virtual Disks

A virtual disk is a storage resource made to look and behave like a complete disk volume when made available or "served" to a host. The host will discover a virtual disk in the same manner as locally-attached physical disks and it can be partitioned, assigned drive letters and used as any standard storage. Virtual disks are created from two types of storage sources: disk pools or pass-through disks.

Virtual disks can be created in logical sizes up to a maximum of 1 petabyte (PB), although the amount of space allocated from the pool will reflect the amount of physical space that has actually been used by the host.

Features

  • Virtual disks utilize high performance caching that increases both transactional speeds and throughput.
  • Three types: single, dual, or mirrored. See Virtual Disks for complete descriptions. Dual virtual disks are created from shared multi-port array pools. See Shared Multi-port Array Support.
  • A mirrored virtual disk provides data redundancy by storing the same data on two DataCore Servers. If a failure occurs on one DataCore Server, then the other DataCore Server takes over immediately to provide continuous data availability and protect against data loss in the event of failures. See Virtual Disks and Mirror Recovery.
  • The Dynamic Data Resiliency feature provides extended protection against failure scenarios by increasing availability and reducing periods without redundancy by adding an additional data copy to mirrored virtual disks. The advanced mirror capabilities of the Dynamic Data Resiliency feature can self-heal the 3-copy virtual disk manually or automatically without intervention and reduce recovery time by maintaining data redundancy in order to avoid full recoveries. See Dynamic Data Resiliency.
  • The Witness feature provides an additional layer of fault tolerance in the event that virtual disks lose all mirror paths and management paths and DataCore Servers are unable to communicate. This feature is particularly useful in clustered environments or when mirrored virtual disks exist in different site locations and inter-site communication fails. See Witness Feature.
  • Mirror recovery has several setting to control the recovery of virtual disk mirrors in order to optimize host performance when necessary. The Mirror Recovery Setting is a global server group setting that controls the overall mirror recovery traffic for all servers in a server group. The Pause/Resume Recovery feature enables virtual disks in recovery to be paused and resumed as necessary. The Auto-recovery setting for virtual disks can be enabled or disabled. See Mirror Recovery.
  • The logstore can optimize mirror recovery by eliminating full mirror recoveries when data changes have been saved to the logstore. See Mirror Recovery.
  • A storage source used to create mirrored virtual disks can be replaced with a new source. A storage source in a mirrored virtual disk can be moved from one pool to another with minimal loss of high availability. See Replacing/Moving a Storage Source in a Virtual Disk.
  • The Maintenance Mode feature consists of two operations which provide a way to evacuate and redistribute virtual disk storage sources created from pools shared by multiple DataCore Servers. This feature moves virtual disk storage sources in a pool from one server to another while maintaining virtual disk high availability. See Maintenance Mode.
  • The storage profile setting of a virtual disk specifies the importance of the data and determines the tier affinity, sets the replication transfer priority and the mirror recovery priority. See Storage Profiles.
  • The I/O paths and LUNs used for virtual disks can be changed. See Modifying Paths.
  • Virtual disk space (comprised of storage allocation units - SAUs) can be reclaimed from virtual disks that have been served to hosts after files have been deleted. See Reclaiming Virtual Disk Space in Pools.
  • Virtual disks can be grouped in order to perform the same operations on all members of the group simultaneously. See Virtual Disk Groups.
  • Quality of Service (QoS) is the management of I/O traffic in order to meet the demands of different applications or to limit utilization by restricting the amount of I/O operations and data transferred for virtual disk groups. QoS settings include limits for the number of I/O operations and the amount of data transferred. See Quality of Service.
  • Virtual disk templates are used to duplicate properties and settings in order to create virtual disks in a standardized fashion. The template contains all basic properties of a virtual disk, including storage sources to use, as well as settings for features and System Health thresholds. Templates can be used to create virtual disks that uniformly fulfill the requirements of applications they are intended to support. See Virtual Disk Templates.
  • The Snapshot feature provides point-in-time copies of virtual disks which can be created while in a known good state. Snapshots can be used to recover from data errors, for application testing, and fast data cloning. See Snapshot for details.
  • The Replication feature provides a solution to asynchronously copy virtual disk data over long distances. Replication can be used for off-site data backup, archiving, emergency site failover for disaster recovery, or controlled site swaps for situations; such as site maintenance, scheduled power cut-off, construction activity, and threat of a natural disaster. See Replication for details.
  • The Continuous Data Protection (CDP) feature saves data changes for a virtual disk in a history log for a period of time. Rollbacks—created from the CDP-enabled virtual disk—can be created at any point in time during the valid time period, and used to recover data. Rollbacks can also be used as temporary backups. See Continuous Data Protection (CDP) for details.
  • The Sequential Storage feature optimizes the data layout of write operations for a virtual disk by storing them sequentially (instead of randomly) which may improve performance for certain storage configurations. This feature is set per virtual disk or virtual disk group. See Sequential Storage.
  • A Windows PowerShell script file named Repair-DcsVirtualDisk.ps1 is provided to repair failed mirrored virtual disks by replacing the failed storage source with a new storage source from a healthy disk pool on any server in the server group. See Repairing Virtual Disks.
  • System health thresholds can be set for virtual disks. See System Health Thresholds.

For more information, see Virtual Disks.

DataCore Cmdlets for Windows PowerShell

DataCore Software brings the features and functionality of DataCore SANsymphony to the Windows PowerShell command line interface.

DataCore Cmdlets for Windows PowerShell Support is installed as a component of DataCore SANsymphony and has over 150 cmdlets so that administrators can manage DataCore SANsymphony in a command line or scripting environment. This feature is supported for use with any DataCore Server running the same version of the software. The software component can also be installed on hosts or computers used for remote management running the DataCore Management Console.

For more information, see Getting Started with DataCore Cmdlets.

VMware vCenter Integration

The VMware vCenter Integration feature integrates DataCore SANsymphony storage services and management with the VMware® vSphere datacenter platform and vCenter management. VMware vCenter Integration automates the process of discovering and adding VMware hosts to the local server group. Virtual disks can be served as storage resources to VMware hosts (ESXi servers). The VMware vCenter Integration feature effectively manages and monitors all virtual hosts and storage resources from the DataCore Management Console. Common storage tasks such as creating snapshots and datastores can be performed.

For more information, see VMware vCenter Integration.

DataCore VASA Provider

DataCore Software integrates DataCore SANsymphony storage services with the virtualization world of VMware vSphere datacenter platform and vCenter™ management by providing a vSphere Aware Storage application programming interface (VASA) Provider which is a set of APIs that integrates storage arrays with vSphere vCenter and allows the capabilities to be known for DataCore SANsymphony . The DataCore VASA Provider allows administrators to configure DataCore SANsymphony virtual disks as storage containers for virtual machines, and monitor topology (devices, paths, adapters), storage capabilities, and health of physical storage used by virtual machines—all from the VMware vSphere Web Client.

The DataCore VASA Provider is installed separately using the Smart Deployment Wizard included in the DataCore SANsymphony installation package. For installation and requirements, see the DataCore VASA Provider Help. For more information, see Getting Started with VASA Provider.

DataCore Storage Management Provider

DataCore Storage Management Provider (SMP) is a WMI-based provider which was designed to support integration and automation between Microsoft System Center Virtual Machine Manager (SCVMM) and the DataCore SANsymphony Software-defined Storage Platform. DataCore Storage Management Provider enables administrators to easily add DataCore SANsymphony server group storage resources to the provider, create logical units (virtual disks), and allocate storage for virtual machines. See Getting Started with DataCore Storage Management Provider.

Diagnostics

Live Performance

  • The Live Performance tool gathers specific performance-related measurements, known as counters, for SAN resources in real-time.
  • The counters are logically grouped by object category, as opposed to drivers, which makes finding all device-related counters much easier and eliminates overlooking counters by users who are not familiar with the drivers.
  • Counter data can be provided full scope for all SAN resources in the same window or specialized for a specific SAN resource.

For more information, see Live Performance.

Recorded Performance

  • The Recorded Performance tool records performance data and stores the data on a DataCore Server in the local server group or a remote SQL Server.
  • Recorded performance data can be selectively retrieved for viewing in the DataCore Management Console or can be output to a CSV file. Data can be retrieved for a specified time period.

For more information, see Recorded Performance.

System Health

  • System Health provides a system wide performance view of SAN resources.
  • Summaries are provided for status of managed resources requiring attention, bandwidth measurements of ports, disk pools and physical disks, and storage capacity breakdowns.
  • When the status of any managed resource changes, it is reported in the tool.
  • Custom threshold settings are provided for several resources that are monitored by System Health. See System Health Thresholds.

For more information, see System Health Tool.

Event Logs

  • DataCore SANsymphony timestamps and records informational, warning and error events as they occur. Configuration changes also produce log messages including the user who performed them as an audit trail.
  • Events can viewed globally for all events in one window or viewed for a specific object.
  • Messages can be sorted, filtered, or searched for specific text.
  • High priority messages will generate alerts. Alerts can also be user-generated in tasks.

For more information, see Event Log and Alerts.

Tasks

  • Tasks is a feature to assist in the administration of DataCore SANsymphony . A task is a set of actions that are performed when one or more triggers occur.
  • Triggers can include a scheduled date/time, receipt of checkpoint markers in replication, a change of state in System Health, or the posting of an event log message.
  • Actions can include snapshot operations, posting an event log message, running a command or Windows PowerShell script file, sending an email notification and reclaiming unused virtual disk space.

For more information, see Automated Tasks.

Reports

  • A report is a "snapshot" of the configuration state at the time it is created. The report tool collects information and data from the system and presents it in a user friendly format. Filters are provided for most reports in order to fine-tune the report output. Report data can be ordered by selecting from available sortable fields and presented in ascending or descending order.

For more information, see Reports Tool.

Support Bundles

  • Support bundles are a collection of system files used to diagnose customer issues should they occur.
  • Support Bundles can be automatically collected from all DataCore Servers in the server group and uploaded from the customer site to DataCore Technical Support for analysis.
  • Support Bundle Relay Service software can be installed as an alternative method for uploading support bundles to DataCore Technical Support when the DataCore Servers in the server group do not have access to the Internet. This separate software package is installed on a machine with Internet access and the server host name or address is configured for the server group.

For more information, see Support Bundles.

SNMP Support

  • DataCore SANsymphony allows SNMP compliant tools to access internal DataCore SANsymphony monitor data and receive trap notifications from DataCore Servers whenever a monitor changes state. Monitors provide basic health information about resources managed by DataCore SANsymphony .

For more information, see SNMP Support.

Integration with Windows Performance Monitor

  • The Integration with Windows Performance Monitor feature exposes DataCore SANsymphony performance counters to the Windows Performance Monitor (perfmon) tool which logs and analyzes system performance.
  • The DataCore Performance Monitor Configuration console is used to select DataCore SANsymphony performance counters, create data collector sets, and export them to Windows Performance Monitor.

For more information, see Integration with Windows Performance Monitor.

DataCore Management Console

The DataCore Management Console centralizes the configuration, allocation, and monitoring of all managed storage resources. This intuitive and flexible user interface makes the configuration and management of storage virtualization simple and easy.

Features

  • Once two or more DataCore Servers are designated as being in the same "server group," all configuration, operations, and management can be performed from either DataCore Server.
  • Configuration and management activities can also be performed remotely from any networked computer with a supported Windows operating system that is running only the DataCore Management Console component.
  • Comprehensive information is presented for all resources managed by DataCore SANsymphony including DataCore Servers, hosts, network connections and ports, physical disks and pools. The console is capable of providing an advanced view of information.
  • Context menus are provided from most locations in the console including panels and lists.
  • All menu command buttons are consolidated in the Ribbon.
  • The console supports "on-the-fly" operations, drag-and-drop capabilities, and multiple selections when performing operations.
  • The different areas of the console can be moved, re-sized, and sorted. For large configuration usability, columns in lists can be manipulated and customized, and a search feature filters out items that do not match the criteria and highlights the matching criteria strings.

For more information, see DataCore Management Console.