Hyper-V – Silverton Consulting

Springpath SDS springs forth

Posted on June 8, 2015June 8, 2015 by Ray in Clustered storage, Data compression, Data reduction, Disk storage, File Storage, SSD storage, Storage, Strategic Inflection Points

Springpath presented at SFD7 and has a new Software Defined Storage (SDS) that attempts to provide the richness of enterprise storage in a SDS solution running on commodity hardware. I would encourage you to watch the SFD7 video stream if you want to learn more about them.

HALO software

Their core storage architecture is called HALO which stands for Hardware Agnostic Log-structured Object store. We have discussed log-structured file systems before. They are essentially a sequential file that can be randomly accessed (read) but are sequentially written. Springpath HALO was written from scratch, operates in user space and unlike many SDS solutions, has no dependencies on Linux file systems.

HALO supports both data deduplication and compression to reduce storage footprint. The other unusual feature is that they support both blade servers and standalone (rack) servers as storage/compute nodes.

Tiers of storage

Each storage node can optionally have SSDs as a persistent cache, holding write data and metadata log. Storage nodes can also hold disk drives used as a persistent final tier of storage. For blade servers, with limited drive slots, one can configure blades as part of a caching tier by using SSDs or PCIe Flash.

All data is written to the (replicated) caching tier before the host is signaled the operation is complete. Write data is destaged from the caching tier to capacity tier over time, as the caching tier fills up. Data reduction (compression/deduplication) is done at destage.

The caching tier also holds read cached data that is frequently read. The caching tier also has a non-persistent segment in server RAM.

Write data is distributed across caching nodes via a hashing mechanism which allocates portions of an address space across nodes. But during cache destage, the data can be independently spread and replicated across any capacity node, based on node free space available. This is made possible by their file system meta-data information.

The capacity tier is split up into data and a meta-data partitions. Meta-data is also present in the caching tier. Data is deduplicated and compressed at destage, but when read back into cache it’s de-compressed only. Both capacity tier and caching tier nodes can have different capacities.

HALO has some specific optimizations for flash writing which includes always writing a full SSD/NAND page and using TRIM commands to free up flash pages that are no longer being used.

HALO SDS packaging under different Hypervisors

In Linux & OpenStack environments they run the whole storage stack in Docker containers primarily for image management/deployment, including rolling upgrade management.

In VMware and HyperVM, Springpath runs as a VM and uses direct path IO to access the storage. For VMware Springpath looks like an NFSv3 datastore with VAAI and VVOL support. In Hyper-V Springpath’s SDS is an SMB storage device.

For KVM its an NFS storage, for OpenStack one can use NFS or they have a CINDER plugin for volume support.

The nice thing about Springpath is you can build a cluster of storage nodes that consists of VMware, HyperV and bare metal Linux nodes that supports all of them. (Does this mean it’s multi protocol, supporting SMB for Hyper-V, NFSv3 for VMware?)

HALO internals

Springpath supports (mostly) file, block (via Cinder driver) and object access protocols. Backend caching and capacity tier all uses a log structured file structure internally to stripe data across all the capacity and caching nodes. Data compression works very well with log structured file systems.

All customer data is supported internally as objects. HALO has a write-log which is spread across their caching tier and a capacity-log which is spread across the capacity tier.

Data is automatically re-balanced across nodes when new nodes are added or old nodes deleted from the cluster.

Data is protected via replication. The system uses a minimum of 3 SSD nodes and 3 drive (capacity) nodes but these can reside on the same servers to be fully operational. However, the replication factor can be configured to be less than 3 if you’re willing to live with the potential loss of data.

Their system supports both snapshots (2**64 times/object) and storage clones for test dev and backup requirements.

Springpath seems to have quite a lot of functionality for a SDS. Although, native FC & iSCSI support is lacking. For a file based, SDS for hypbervisors, it seems to have a lot of the bases covered.

Comments?

Windows Server 2012 R2 storage changes announced at TechEd

Posted on June 6, 2013September 8, 2013 by Ray in CIFS/SMB, Cloud storage, DAS, desktop virtualization, Ethernet, File Storage, RDMA, RoCE, Server virtualization, SSD storage, Storage performance

Microsoft TechEd USA is this week and they announced a number of changes to the storage services that come with Windows Server 2012 R2

Azure DRaaS – Microsoft is attempting to democratize DR by supporting a new DR-as-a-Service (DRaaS). They now have an Azure service that operates in conjunction with Windows Server 2012 R2 that provides orchestration and automation for DR site failover and fail back to/from remote sites. Windows Server 2012 R2 uses Hyper-V Replica to replicate data across to the other site. Azure DRaaS supports DR plans (scripts) to identify groups of Hyper-V VMs which need to be brought up and their sequencing. VMs within a script group are brought up in parallel but different groups are brought up in sequence. You can have multiple DR plans, just select the one to execute. You must have access to Azure to use this service. Azure DR plans can pause for manual activities and have the ability to invoke PowerShell scripts for more fine tuned control. There’s also quite a lot of setup that must be done, e.g. configure Hyper-V hosts, VMs and networking at both primary and secondary locations. Network IP injection is done via mapping primary to secondary site IP addresses. The Azzure DRaaS really just provides the orchestration of failover or fallback activity. Moreover, it looks like Azure DRaaS is going to be offered by service providers as well as private companies. Currently, Azure’s DRaaS has no support for SAN/NAS replication but they are working with vendors to supply an SRM-like API to provide this.
Hyper-V Replica changes – Replica support has been changed from a single fixed asynchronous replication interval (5 minutes) to being able to select one of 3 intervals: 15 seconds; 5 minutes; or 30 minutes.
Storage Spaces Automatic Tiering – With SSDs and regular rotating disk in your DAS (or JBOD) configuration , Windows Server 2012 R2 supports automatic storage tiering. At Spaces configuration time one dedicates a certain portion of SSD storage to tiering. There is a scheduled Windows Server 2012 task which is then used to scan the previous periods file activity and identify which file segments (=1MB in size) that should be on SSD and which should not. Then over time file segments are moved to an appropriate tier and then, performance should improve. This only applies to file data and files can be pinned to a particular tier for more fine grained control.
Storage Spaces Write-Back cache – Another alternative is to dedicate a certain portion of SSDs in a Space to write caching. When enabled, writes to a Space will be cached first in SSD and then destaged out to rotating disk. This should speed up write performance. Both write back cache and storage tiering can be enabled for the same Space. But your SSD storage must be partitioned between the two. Something about funneling all write activity to SSDs just doesn’t make sense to me?!
Storage Spaces dual parity – Spaces previously supported mirrored storage and single parity but now also offers dual parity for DAS. Sort of like RAID6 in protection but they didn’t mention the word RAID at all. Spaces dual parity does have a write penalty (parity update) and Microsoft suggests using it only for archive or heavy read IO.
SMB3.1 performance improvements of ~50% – SMB has been on a roll lately and R2 is no exception. Microsoft indicated that SMB direct using a RAM DISK as backend storage can sustain up to a million 8KB IOPS. Also, with an all-flash JBOD, using a mirrored Spaces for backend storage, SMB3.1 can sustain ~600K IOPS. Presumably these were all read IOPS.
SMB3.1 logging improvements – Changes were made to SMB3.1 event logging to try to eliminate the need for detail tracing to support debug. This is an ongoing activity but one which is starting to bear fruit.
SMB3.1 CSV performance rebalancing – Now as one adds cluster nodes, Cluster Shared Volume (CSV) control nodes will spread out across new nodes in order to balance CSV IO across the whole cluster.
SMB1 stack can be (finally) fully removed – If you are running Windows Server 2012, you no longer need to install the SMB1 stack. It can be completely removed. Of course, if you have some downlevel servers or clients you may want to keep SMB1 around a bit longer but it’s no longer required for Server 2012 R2.
Hyper-V Live Migration changes – Live migration can now take advantage of SMB direct and its SMB3 support of RDMA/RoCE to radically speed up data center live migration. Also, Live Migration can now optionally compress the data on the current Hyper-V host, send compressed data across the LAN and then decompress it at target host. So with R2 you have three options to perform VM Live Migration traditional, SMB direct or compressed.
Hyper-V IO limits – Hyper-V hosts can now limit the amount of IOPS consumed by each VM. This can be hierarchically controlled providing increased flexibility. For example one can identify a group of VMs and have a IO limit for the whole group, but each individual VM can also have an IO limit, and the group limit can be smaller than the sum of the individual VM limits.
Hyper-V supports VSS backup for Linux VMs – Windows Server 2012 R2 has now added support for non-application consistent VSS backups for Linux VMs.
Hyper-V Replica Cascade Replication – In Windows Server 2012, Hyper V replicas could be copied from one data center to another. But now with R2 those replicas at a secondary site can be copied to a third, cascading the replication from the first to the second and then the third data center, each with their own replication schedule.
Hyper-V VHDX file resizing – With Windows Server 2012 R2 VHDX file sizes can now be increased or reduced for both data and boot volumes.
Hyper-V backup changes – In previous generations of Windows Server, Hyper-V backups took two distinct snapshots, one instantaneously and the other at quiesce time and then the two were merged together to create a “crash consistent” backup. But with R2, VM backups only take a single snapshot reducing overhead and increasing backup throughput substantially.
NVME support – Windows Server 2012 R2 now ships with a Non-Volatile Memory Express (NVME) driver for PCIe flash storage. R2’s new NVME driver has been tuned for low latency and high bandwidth and can be used for non-clustered storage spaces to improve write performance (in a Spaces write-back cache?).
CSV memory read-cache – Windows Server 2012 R2 can be configured to set aside some host memory for a CSV read cache. This is different than the Spaces Write-Back cache. CSV caching would operate in conjunction with any other caching done at the host OS or elsewhere.

That’s about it. Some of the MVPs had a preview of R2 up in Redmond, but all of this was to be announced in TechEd, New Orleans, this week.

~~~~

Image: Microsoft TechEd by BetsyWeber

NetApp and Microsoft ink 3 year agreement

Posted on December 10, 2009 by Ray in Strategy

Microsoft logo

NetApp Logo

This week, NetApp and Microsoft announced a new agreement that increases the collaboration and integration for both their product technology and sales&marketing activities. Specifically, the new agreement covers better integration of

Microsoft’s virtualized infrastructure interaction with NetApp storage. This currently consists of new PRO Packs for System Center to support NetApp storage and Snap Manager for Hyper-V. Better integration such as these between these Microsoft virtualization and NetApp storage should help ease administration and maximize utilization of customer storage for Hyper-V environments.
Microsoft Exchange, SQL and Office SharePoint servers with NetApp Storage. This includes new support for Exchange 2010 in NetApp Snap Manager that fully provides deduplication and replication of Exchange data. Further integration along these lines should lead to even better customer utilization of storage in these environments in the near future.
Microsoft dynamic data center initiative and NetApp storage. This includes better testing and compatibility for using NetApp storage under Microsoft’s Dynamic Data Center Toolkit for enterprise private clouds (DDTK-E). Better integration and support for Microsoft’s Dynamic Data Center by NetApp storage and vice versa should help customers build more storage efficient cloud infrastructure.
Microsoft and NetApp joint go to market activities. This will result in Microsoft and NetApp joint channel partners being better able to offer joint solutions to their customers and also, customers should see better collaboration and partnership between NetApp and Microsoft sales and support.

This agreement should be considered a deepening of NetApp and Microsoft’s ongoing alliance. How this will actually change either company’s product functionality in the future was left unsaid but one should see more of NetApp’s advanced storage features show up as being available directly from Microsoft systems and NetApp storage services tighter integration with Microsoft systems should make it easier for customers to use storage more efficiently. Exactly when these activities result in enhanced NetApp or Microsoft functionality was not stated and is no doubt subject of ongoing discussions between the two teams. Nonetheless, this agreement seems to say that NetApp and Microsoft are taking their alliance to the next level.

Full disclosure: I am currently working on a contract with NetApp on another aspect of their storage but am doing no work with Microsoft.