Shabeera et al., 2013 - Google Patents

Bandwidth-aware data placement scheme for Hadoop

Shabeera et al., 2013

Document ID
3483980550007915762
Author
Shabeera T
Kumar S
Publication year
Publication venue
2013 IEEE Recent Advances in Intelligent Computational Systems (RAICS)

External Links

Snippet

We are living in a data rich era. The size of the data is increasing exponentially. Social networking applications, Scientific experiments, etc. are the major contributors of Big Data. The data can be structured, semi-structured or unstructured. Big Data management solutions …
Continue reading at ieeexplore.ieee.org (other versions)

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30067—File systems; File servers
    • G06F17/30129—Details of further file system functionalities
    • G06F17/3015—Redundancy elimination performed by the file system
    • G06F17/30156—De-duplication implemented within the file system, e.g. based on file segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30286—Information retrieval; Database structures therefor; File system structures therefor in structured data stores
    • G06F17/30575—Replication, distribution or synchronisation of data between databases or within a distributed database; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for programme control, e.g. control unit
    • G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30943—Information retrieval; Database structures therefor; File system structures therefor details of database functions independent of the retrieved data type
    • G06F17/30946—Information retrieval; Database structures therefor; File system structures therefor details of database functions independent of the retrieved data type indexing structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from or digital output to record carriers, e.g. RAID, emulated record carriers, networked record carriers
    • G06F3/0601—Dedicated interfaces to storage systems
    • G06F3/0628—Dedicated interfaces to storage systems making use of a particular technique
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from or digital output to record carriers, e.g. RAID, emulated record carriers, networked record carriers
    • G06F3/0601—Dedicated interfaces to storage systems
    • G06F3/0602—Dedicated interfaces to storage systems specifically adapted to achieve a particular effect
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network-specific arrangements or communication protocols supporting networked applications
    • H04L67/10—Network-specific arrangements or communication protocols supporting networked applications in which an application is distributed across nodes in the network
    • H04L67/104—Network-specific arrangements or communication protocols supporting networked applications in which an application is distributed across nodes in the network for peer-to-peer [P2P] networking; Functionalities or architectural details of P2P networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network-specific arrangements or communication protocols supporting networked applications
    • H04L67/10—Network-specific arrangements or communication protocols supporting networked applications in which an application is distributed across nodes in the network
    • H04L67/1097—Network-specific arrangements or communication protocols supporting networked applications in which an application is distributed across nodes in the network for distributed storage of data in a network, e.g. network file system [NFS], transport mechanisms for storage area networks [SAN] or network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F15/00—Digital computers in general; Data processing equipment in general
    • G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a programme unit and a register, e.g. for a simultaneous processing of several programmes

Similar Documents

Publication Publication Date Title
Wei et al. CDRM: A cost-effective dynamic replication management scheme for cloud storage cluster
Bai et al. RTRM: A response time-based replica management strategy for cloud storage system
Li et al. A convergence of key‐value storage systems from clouds to supercomputers
Xu et al. Building a large-scale object-based active storage platform for data analytics in the internet of things
Chen et al. E2FS: an elastic storage system for cloud computing
Shabeera et al. Bandwidth-aware data placement scheme for Hadoop
US20170220586A1 (en) Assign placement policy to segment set
Singh et al. Scalable metadata management techniques for ultra-large distributed storage systems--A systematic review
Mansouri QDR: a QoS-aware data replication algorithm for Data Grids considering security factors
Dev et al. Har+: Archive and metadata distribution! why not both?
Caneill et al. Locality-aware routing in stateful streaming applications
Chihoub et al. Exploring energy-consistency trade-offs in cassandra cloud storage system
Dev et al. Performance evaluation of HDFS in big data management
Zhao et al. Dynamic replica creation strategy based on file heat and node load in hybrid cloud
Zhou et al. Hierarchical consistent hashing for heterogeneous object-based storage
Lu et al. Hybrid storage architecture and efficient MapReduce processing for unstructured data
CN103744882A (en) Catalogue fragment expressing method and device based on key value pair
Cha et al. Adaptive metadata rebalance in exascale file system
Zhang et al. Key based data analytics across data centers considering bi-level resource provision in cloud computing
Tao et al. A dynamic data allocation method with improved load-balancing for cloud storage system
Luo et al. Data placement algorithm for improving I/O load balance without using popularity information
KR101512647B1 (en) Method For Choosing Query Processing Engine
Singh et al. High scalability of HDFS using distributed namespace
Huang et al. Resource provisioning with QoS in cloud storage
Meng et al. A network load sensitive block placement strategy of HDFS