Scality’s Open Source S3 Driver

img_6931
The view from Scality’s conference room

We were at Scality last week for Cloud Field Day 1 (CFD1) and one of the items they discussed was their open source S3 driver. (Videos available here).

Scality was on the 25th floor of a downtown San Francisco office tower. And the view outside the conference room was great. Giorgio Regni, CTO, Scality, said on the two days a year it wasn’t foggy out, you could even see Golden Gate Bridge from their conference room.

Scality

img_6912As you may recall, Scality is an object storage solution that came out of the telecom, consumer networking industry to provide Google/Facebook like storage services to other customers.

Scality RING is a software defined object storage that supports a full complement of interface legacy and advanced protocols including, NFS, CIGS/SMB, Linux FUSE, RESTful native, SWIFT, CDMI and Amazon Web Services (AWS) S3. Scality also supports replication and erasure coding based on object size.

RING 6.0 brings AWS IAM style authentication to Scality object storage. Scality pricing is based on usable storage and you bring your own hardware.

Giorgio also gave a session on the RING’s durability (reliability) which showed they support 13-9’s data availability. He flashed up the math on this but it was too fast for me to take down:)

Scality has been on the market since 2010 and has been having a lot of success lately, having grown 150% in revenue this past year. In the media and entertainment space, Scality has won a lot of business with their S3 support. But their other interface protocols are also very popular.

Why S3?

It looks as if AWS S3 is becoming the defacto standard for object storage. AWS S3 is the largest current repository of objects. As such, other vendors and solution providers now offer support for S3 services whenever they need an object/bulk storage tier behind their appliances/applications/solutions.

This has driven every object storage vendor to also offer S3 “compatible” services to entice these users to move to their object storage solution. In essence, the object storage industry, like it or not, is standardizing on S3 because everyone is using it.

But how can you tell if a vendor’s S3 solution is any good. You could always try it out to see if it worked properly with your S3 application, but that involves a lot of heavy lifting.

However, there is another way. Take an S3 Driver and run your application against that. Assuming your vendor supports all the functionality used in the S3 Driver, it should all work with the real object storage solution.

Open source S3 driver

img_6916Scality open sourced their S3 driver just to make this process easier. Now, one could just download their S3server driver (available from Scality’s GitHub) and start it up.

Scality’s S3 driver runs ontop of a Docker Engine so to run it on your desktop you would need to install Docker Toolbox for older Mac or Windows systems or run Docker for Mac or Docker for Windows for newer systems. (We also talked with Docker at CFD1).

img_6933Firing up the S3server on my Mac

I used Docker for Mac but I assume the terminal CLI is the same for both.Downloading and installing Docker for Mac was pretty straightforward.  Starting it up took just a double click on the Docker application, which generates a toolbar Docker icon. You do need to enter your login password to run Docker for Mac but once that was done, you have Docker running on your Mac.

Open up a terminal window and you have the full Docker CLI at your disposal. You can download the latest S3 Server from Scality’s Docker hub by executing  a pull command (docker pull scality/s3server), to fire it up, you need to define a new container (docker run -d –name s3server -p 8000:8000 scality/s3server) and then start it (docker start s3server).

It’s that simple to have a S3server running on your Mac. The toolbox approach for older Mac’s and PC’S is a bit more complicated but seems simple enough.

The data is stored in the container and persists until you stop/delete the container. However, there’s an option to store the data elsewhere as well.

I tried to use CyberDuck to load some objects into my Mac’s S3server but couldn’t get it to connect properly. I wrote up a ticket to the S3server community. It seemed to be talking to the right port, but maybe I needed to do an S3cmd to initialize the bucket first – I think.

[Update 2016Sep19: Turns out the S3 server getting started doc said you should download an S3 profile for Cyberduck. I didn’t do that originally because I had already been using S3 with Cyberduck. But did that just now and it now works just like it’s supposed to. My mistake]

~~~~

Anyways, it all seemed pretty straight forward to run S3server on my Mac. If I was an application developer, it would make a lot of sense to try S3 this way before I did anything on the real AWS S3. And some day, when I grew tired of paying AWS, I could always migrate to Scality RING S3 object storage – or at least that’s the idea.

Comments?

Rubrik has a better idea for VMware backup

Cluster nodesRubrik has been around since January 2014 and just GA’d in April of last year. They recently presented at  TechFieldDay 10 (TFD10, videos here) with Chris Wahl, Technical Evangelist, Arvin “Nitro” Nithrakashyap, Co-Founder and Bipul Sinha, Co-Founder, in attendance.

I have known Chris Wahl since November of 2013, from our time together on Storage Field Day 4 (SFD4). Howard and I (the “Greybeards”) also interviewed Chris Wahl for Rubrik on a Greybeards on Storage podcast.
Continue reading “Rubrik has a better idea for VMware backup”

Object store and hybrid clouds at Cloudian

IMG_4364Out of Japan comes another object storage provider called Cloudian.  We talked with Cloudian at Storage Field Day 7 (SFD7) last month in San Jose (see the videos of their presentations here).

Cloudian history

Cloudian has been out on the market since March of 2011 but we haven’t heard much about them, probably because their focus has been East Asia.  The same day that the  Tōhoku Earthquake and Tsunami hit the company announced Cloudian, an Amazon S3 Compliant Multi-Tenant Cloud Storage solution.

Their timing couldn’t have been better. Japanese IT organizations were beating down their door over the next two years for a useable and (earthquake and tsunami) resilient storage solution.

Cloudian spent the next 2 years, hardening their object storage system, the HyperStore, and now they are ready to take on the rest of the world.

Currently Cloudian has about 20PB of storage under management and are shipping a HyperStore Appliance or a software only distribution of their solution. Cloudian’s solutions support S3 and NFS access protocols.

Their solution uses Cassandra, a highly scaleable, distributed NoSQL database which came out of FaceBook for their meta-data database. This provides a scaleable, non-sharable meta-data data base for object meta-data repository and lookup.

Cloudian creates virtual storage pools on backend storage which can be optimized for small objects, replication or erasure coding and can include automatic tiering to any Amazon S3 and Glacier compatible cloud storage. I would guess this is how they qualify for Hybrid Cloud status.

The HyperStore appliance

Cloudian creates a HyperStore P2P ring structure. Each appliance has Cloudian management console services as well as the HyperStore engine which supports three different data stores: Cassandra, Replicas, and Erasure coding. Unlike Scality, it appears as if one HyperStore Ring must exist in a region. But it can be split across data centers. Unclear what their definition of a “region” is.

HyperStore hardware come in entry level (HSA-2024: 24TB/1U), capacity optimized (HSA-2048: 48TB/1U), performance optimized (HSA-2060: all flash, 60TB/2U

Replication with Dynamic Consistency

The other thing that Cloudian supports is different levels of consistency for replicated data. Most object stores support eventual consistency (see Eventual Data Consistency and Cloud Storage post).  HyperStore supports 3 (well maybe 5) different levels of consistency:

  1. One – object written to one replica and committed there before responding to client
  2. Quorum – object written to N/2+1 replicas before responding to client
    1. Local Quorum – replicas are written to N/2+1 nodes in same data center  before responding to client
    2. Each Quorum – replicas are written to N/2+1 nodes in each data center before responding to client.
  3. All – all replicas must have received and committed the object write before responding to client

There are corresponding read consistency levels as well. The objects writes have a “coordinator” node which handles this consistency. The implication is that consistency could be established on an object basis. Unclear to me whether Read and Write dynamic consistency can be different?

Apparently small objects are also stored in the  Cassandra datastore.  That way HyperStore optimizes for object size. Also, HyperStore nodes can be added to a ring and the system will auto balance the data across the old and new nodes automatically.

Cloudian also support object versioning, ACL, and QoS services as well.

~~~

I was a bit surprised by Cloudian. I thought I knew all the object storage solutions out on the market. But then again they made their major strides in Asia and as an on-premises Amazon S3 solution, rather than a generic object store.

For more on Cloudian from SFD7 see:

Cloudian – Storage Field Day 7 Preview by @VirtualizedGeek (Keith Townsend)

Object Storage Summit wrap up

Attended ExecEvent’s first Next-Gen Object Storage Summit in Miami this past week.  Learned a lot there and met a lot of the players and movers in this space.  Here is a summary of what happened during the summit.

Janae starting a debate on Object Storage
Janae starting a debate on Object Storage

Spent most of the morning of the first day discussing some parameters of  object storage in general. Janae got up and talked about 4 major adopters for object storage:

  1. Rapid Responders – these customer have data in long term storage and it  just keeps building and needs to be stored in scaleable storage. They believe someday they  will need access to it and have no idea when. But when they want it, they want it fast. Rapid responder adoption  is based on the unpredictability of access. As such, having the data on scaleable disk object storage makes sense.  Some examples include black operations sites with massive surveillance feeds which maybe needed fast sometime after initial analysis and medical archives.
  2. Distributed (content) Enterprises – geographically distributed enterprises with users around the globe that need shared access to data.  Distributed enterprises often have 100 or so users sharing data access dispersed around the globe and want shared access to data.   Object storage can dispurse the data to provide local caching across the world for better data and meta-data latency.  Media and Entertainment are key customers in this space but design shops that follow the sun also have the problem.
  3. Private Cloud(y) – data centers adopt the cloud for a number of reasons but sometimes it’s just mandated.  In these cases, direct control over cloud storage with the economics of major web service providers can be an alluring proposition.  Some object storage solutions roll in with cloud like economics and on premises solutions and responsiveness, the best of all worlds.  Enterprise IT forced to move to the cloud are in this category.
  4. Big Hadoop(ers) – lots of data to analyze but with no understanding of when it will be analyzed.  Some Hadoopers can schedule analytics but most don’t know what they will want until they finish with the last analysis. In these cases, having direct access to all the data on an object store can cut setup time considerably.

There were other aspects of Janae’s session but these seemed of most interest. We spent the debating aspects of object storage rest of the morning getting an overview on Scality customers. At the end of the morning we debating aspects of object storage.  I thought Jean-Luc from Data Direct Networks had the best view of this when he said object storage is at it’s core, data storage that has scalability, resilience, performance and distribution.

The afternoon sessions were deep dives with the sponsors of the Object Summit.

  • Nexsan talked about there Assureon product line (EverTrust acquisition).  SHA1 and MD5 hashes are made of every object then as objects are replicated to other sites, the hashes are both checked to insure the data hasn’t been corrupted and the are  periodically checked (every 90 days) to see if the data is still correct. If it’s corrupted,  other replica’s obtained and re-instated.  In addition, Assureon has some unique immutable access logs that provide an almost “chain of custody” for objects in the system.  Finally, Assureon uses a Microsoft Windows Agent that is Windows Certified and installs without disruption to allow any user (or administrator) to identify files, directories, or file systems to be migrated to the object store.
  • Cleversafe was up next and talked about their market success with their distributed dsNet® object store and provided some proof points. [Full disclosure: I have recently been under contract with Cleversafe]. For instance, today they have under management over 15 billion objects and deployments with over 70PBs in production They have shipped over 170PB of dsNet storage to customers around the world. Cleversafe has many patents covering their information dispersal algorithms and performance optimization.  Some of their sites are in the Federal government installations with a few web intensive clients as well, the most notable being Shutterfly, photo sharing site.  Although dsNet is inherently geographical distributed  all these “sites” could easily be configured over 1 to 3 locations or more for simpler DR-like support.
  • Quantum talked about their Lattus product  built ontop of Amplidata’s technology. Lattus uses 36TB storage nodes, controller nodes to provide erasure coding for geographical data integrity and NAS gateway nodes.  The NAS gateway provides CIFS and NFS to objects. The Latus-C deployment is a forever disk archive for cloud like deployments. This system provides erasure coding for objects in the system which are then dispersed across up to 3 sites (today, with 4 site dispersal under test).  Their roadmap Lattus-M is going to be a managed file system offering that operates in conjunction with their StorNext product with ILMlike policy management. Farther out, on the roadmap is a Lattus-H which offers object repository for Hadoop clusters that can gain rapid access to data for analysis.
  • Scality talked about their success in major multi-tennant environments that need rock-solid reliability and great performance. Their big customers are major web providers that supply email services. Scality is a software product that builds a ring of object storage nodes that supplies the backend storage where the email data is held.  Scality is priced on a per end-user capacity stored. Today the product supports RestFul interfaces, CDMI (think email storage interface), Scality File System (based on FUSE, a POSIX compliant Linux file system). NFS interface is coming early next year.  With the Scality Ring, nodes can go down but the data is still available with rapid response time.  Nodes can be replicated or spread across multiple locations
  • Data Direct Networks (DDN) is coming at the problem from the High Performance Computing market and have an very interesting scaleable solution with extreme performance. DDN products are featured in many academic labs and large web 2.0 environments.  The WOS object storage supports just about any interface you want Java, PHP, Python, RestFULL, NFS/CIFS, S3 and others. They claim very high performance something on the order of 350MB/sec read and 250MB/sec write (I think per node) of object data transfers.  Nodes come in 240TB units and one can have up to 256 nodes in a WOS system.   One customer uses a WOS node to land local sensor streams then ships it to other locations for analysis.
View from the Summit balcony, 2nd day
View from the Summit balcony, 2nd day

The next day was spent with Nexsan and DDN talking about their customer base and some of their success stories. We spent the remainder of the morning talking about the startup world which surrounds some object storage technology and the inhibiters to broader adoption of the technology.

In the end there’s a lot of education needed to jump start this market place. Education about both the customer problems that can be solved with object stores and the product differences that are out there today.  I argued (forcefully) that what’s needed to accelerate adoption was some standard interface protocol that all object storage systems could utilize. Such a standard protocol would enable a more rapid ecosystem build out and ultimately more enterprise adoption.

One key surprise to me was that the problems their customers are seeing is something all IT customers will have some day. Jean-Luc called it the democratization of the HPC problems. Big Data is driving object storage requirements into the enterprise in a big way…

Comments?

Initial impressions on Spring SNW/Santa Clara

I heard storage beers last nite was quite the party, sorry I couldn’t make it but I did end up at the HDS customer reception which was standing room only and provided all the food and drink I could consume.

Saw quite a lot of old friends too numerous to mention here but they know who they are.

As for technology on display there was some pretty impressive stuff.

Verident card (c) 2011 Silverton Consulting, Inc.
Verident card (c) 2011 Silverton Consulting, Inc.

Lots of great technology on display there.

Virident tachIOn SSD

One product that caught my eye was from Virident, their tachIOn SSD. I called it a storage subsystem on a board.  I had never talked with them before but they have been around for a while using NOR storage but now are focused on NAND.

Their product is a fully RAIDed storage device using flash aware RAID 5 parity locations, their own wear leveling and other SSD control software and logic with replaceable NAND modules.

Playing with this device I felt like I was swapping drives of the future. Each NAND module stack has a separate controller and supports high parallelism.  Talking with Shridar Subramanian, VP of marketing, he said the product is capable of over 200K IOPS running a fully 70% read:30% write workload at full capacity.

They have a Capacitor backed DRAM buffer which is capable of uploading the memory buffer to NAND after a power failure. It plugs into a PCIe slot and uses less than 25W of power, in capacities of 300-800GB.  It requires a software driver, they currently only support Linux and VMware (a Linux varient) but Windows and other O/Ss are on the way

Other SSDs/NAND storage

Their story was a familair refrain throughout the floor, lots of SSD/NAND technology coming out, in various formfactors.  I saw one system using SSDs from Viking Modular Systems that fit into a DRAM DIMM slot and supported a number of SSDs behind a SAS like controller. Also requiring a SW driver.

(c) 2011 Silverton Consulting, Inc.
(c) 2011 Silverton Consulting, Inc.

Of course TMS, Fusion-IO, Micron, Pliant and others were touting their latest SSD/Nand based technology showing off their latest solutions and technology.   For some reason lots of SSD’s at this show.

Naturally, all the other storage vendors were there Dell, HDS, HP, EMC, NetApp and IBM. IBM was showing off Watson, their new AI engine that won at Jeopardy.

And then there was cloud, …

Cloud was a hot topic as well. Saw one guy in the corner I have talked about before StorSimple which is a cloud gateway provider.  They said they are starting to see some traction in the enterprise. Apparently enterprise are starting to adopt cloud – who knew?

Throw in a few storage caching devices, …

Then of course there was the data caching products which ranged from the relaunched DataRAM XcelASAN to Marvel’s new DragonFLY card.  DragonFLY provides a cache on a PCI-E card which DataRAM is a FC caching appliance, all pretty interesting.

… and what’s organic storage?

And finally, Scality came out of the shadows with what they are calling an organic object storage device.  The product reminded me of Bycast (now with NetApp) and Archivas (now with HDS) in that they had a RAIN architecture, with mirrored data in an object store interface.  I asked them what makes them different and Jerome Lecat, CEO said they are relentlessly focused on performance and claims they can retrieve an object in under 40msec.  My kind of product.  I think they deserve a deeper dive sometime later.

—-

Probably missed a other  vendors but these are my initial impressions.  For some reason I felt right at home swapping NAND drive modules,…

Comments