|
|
Starfish Distributed FilesystemFrom DBWiki
[edit] Introduction Starfish is a new type of clustered file storage system that keeps very large amounts of data safe and accessible - even in the face of massive hard drive and storage node failures. The system is free for use if your storage requirements are less than 1 Terabyte. To be more specific, Starfish is a highly-available, fully decentralized, clustered storage file system. It provides a distributed POSIX-compliant storage device that can be mounted like any other drive under Linux or Mac OS X. The resulting fault-tolerant storage network can store files and directories, like a normal file system - but unlike a normal file system, it can handle multiple catastrophic disk and machine failures. Starfish is priced very competitively for storage systems greater than 1TB - a typical system is usually 1/2 to 1/10th of the cost of other proprietary solutions. For example, a 10TB proprietary storage solution costs roughly $150,000 per year to operate. An equivalent 10TB Starfish solution can cost as little as $12,000 per year to operate. The storage network automatically provides file mirroring, disaster recovery, and high-availability in the event of multiple storage or metadata node failures. Like the starfish, this file system can be damaged severely, continue to function, and regrow itself over time. More information regarding Starfish:
[edit] Is Starfish Right for You? Starfish is designed for highly-available services such as web server farms, enterprise-wide data storage and retrieval, digital media storage systems, computing clusters, or long-term data storage. If your storage solution uses Windows CIFS, Samba or NFS - Starfish might be a good fit for your needs. The Starfish Clustered Distributed Filesystem is currently used by Digital Bazaar to store multiple terabytes worth of digital content. One terabyte is roughly 125,000 high quality MP3s, or 210 uncompressed DVDs. Content companies are charged with storing a great deal of digital content produced every year. For example, a single television station can produce close to 10 Terabytes of video footage a year. After 3 years, you can have quite a large storage nightmare on your hands. To make matters worse, inexpensive data backup systems do not exist to deal with this sort of data backup problem. As digital video and audio become more prevalent, companies will have to find a solution for storing massive amounts of data for long periods of time. A storage system that addresses these problems must be easy to setup, administer and roll forward. The storage system must also be tolerant of failures and achievable with very modest IT budgets. Starfish is such a networked storage system. [edit] Goals of the Starfish FilesystemThe goal of the Starfish Filesystem is to solve the problems of scalability, cost, ease of use and high-availability in massive storage systems. [edit] Scalability Starfish was designed to scale linearly from the beginning. For example, if more storage is needed, all one must do to add another 1TB of storage is add another server to the storage cluster ($1,000 as of Jan. 2007). The storage network will automatically detect and bring the new storage peer into the cluster. Typically, to expand a proprietary storage cluster, the cost is around $5,000 to $10,000 per server. [edit] Ease of UseStarfish is auto-configuring - the only pieces of information a starfish peer needs is:
File mirroring is performed automatically, so is file recovery, re-joining an active storage network, and synchronizing with storage peers. [edit] CostClusters are increasingly being built from commodity components - proprietary solutions are not providing a high enough price/performance ratio. There are a number of large companies that use commodity-grade components in their IT infrastructure - Google being one of the biggest. Hardware fails, it is a fact of life. The key is to make the software smart enough to detect failure and continue to operate regardless of failures. Starfish embraces the approach of having non-homogenized, commodity servers running fault-tolerant software to provide a storage solution. By taking this approach, costs are reduced to mere fractions of the cost of a proprietary solution. [edit] High-Availability High-availability means the ability for a system to be available most, if not all, of the time - regardless of numerous sub-system failures. Starfish accomplishes high-availability by being up to N-way redundant to failure scenarios. This means, no centralized metadata servers, no central switching points. Instead Starfish uses a distributed P2P-based storage and metadata architecture. Storage peers can join and leave the network without severely impacting the operation of the storage cluster. This feature comes in handy if parts of your cluster are temporarily taken offline for maintenance purposes. The storage cluster can lose all but one storage node without failing completely. That means that if there is at least one node remaining, the Starfish Filesystem will respond to file system requests that it can serve with the remaining storage node. This is important if you don't want your applications to fail if the data it needs is available on an unaffected portion of the cluster. [edit] Current Features
[edit] Future Features
[edit] DownloadYou can download the Starfish software and source code from our Starfish Software Download page. [edit] Frequently Asked QuestionsYou may peruse the Starfish Frequently Asked Questions list to see if there are any questions that you have that have been answered. [edit] LicenseIt is our intention to release older versions of the Starfish File system under an Open Source (OSI-approved) license. More than likely, we will use the Adaptive Public License. Our customers pay for stable, optimized releases of the Starfish Storage Network Software. For now, the Starfish File system License allows you to do the following:
You must pay for the software if:
You can contact us if you are interested in licensing the Starfish filesystem for unlimited use. The cost is around 1/2 to 1/10th of the cost of all current solutions on the market - there is preferential pricing for start-up companies, non-profits and academic institutions. Support contracts are also available if you would like to extend Starfish to meet your business needs, or require help designing or configuring your Starfish file system solution. For specifics, please refer to the full text of the Starfish Filesystem License. [edit] ContributingIf you would like to send us performance information, patches, bug reports, or suggestions - there are several ways you can do so.
|
