Class LocalFsBlobStore
- All Implemented Interfaces:
AutoCloseable,Shutdownable
For a local blob store the user and the supervisor use NimbusBlobStore Client API in order to talk to nimbus through thrift. The authentication and authorization here is based on the subject. We currently have NIMBUS_ADMINS and SUPERVISOR_ADMINS configuration. NIMBUS_ADMINS are given READ, WRITE and ADMIN access whereas the SUPERVISOR_ADMINS are given READ access in order to read and download the blobs form the nimbus.
The ACLs for the blob store are validated against whether the subject is a NIMBUS_ADMIN, SUPERVISOR_ADMIN or USER who has read, write or admin privileges in order to perform respective operations on the blob.
For local blob store 1. The USER interacts with nimbus to upload and access blobs through NimbusBlobStore Client API. 2. The USER sets the ACLs, and the blob access is validated against these ACLs. 3. The SUPERVISOR interacts with nimbus through the NimbusBlobStore Client API to download the blobs. The supervisors principal should match the set of users configured into SUPERVISOR_ADMINS. Here, the PrincipalToLocalPlugin takes care of mapping the principal to user name before the ACL validation.
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.storm.blobstore.BlobStore
BlobStore.BlobStoreFileInputStream, BlobStore.BlobStoreFileOutputStream, BlobStore.KeyTranslationIterator -
Field Summary
FieldsFields inherited from class org.apache.storm.blobstore.BlobStore
BASE_BLOBS_DIR_NAME -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionbooleanvoidcheckForBlobUpdate(String key) createBlob(String key, SettableBlobMeta meta, Subject who) Creates the blob.voiddeleteBlob(String key, Subject who) Deletes the blob data and metadata.voidfullCleanup(long age) Gets the InputStream to read the blob details.getBlobMeta(String key, Subject who) Gets the current version of metadata for a blob to be viewed by the user or downloaded by the supervisor.intgetBlobReplication(String key, Subject who) Gets the replication factor of the blob.listKeys()Returns an iterator with all the list of keys currently available on the blob store.voidprepare(Map<String, Object> conf, String overrideBase, NimbusInfo nimbusInfo, ILeaderElector leaderElector) Allows us to initialize the blob store.voidsetBlobMeta(String key, SettableBlobMeta meta, Subject who) Sets the metadata with renewed acls for the blob.voidsetLeaderElector(ILeaderElector leaderElector) Sets leader elector (only used by LocalFsBlobStore to help sync blobs between Nimbi.voidshutdown()voidStart the syncing blobs between the local running instance of the BlobStore and others.updateBlob(String key, Subject who) Updates the blob data.intupdateBlobReplication(String key, int replication, Subject who) Modifies the replication factor of the blob.Methods inherited from class org.apache.storm.blobstore.BlobStore
close, createBlob, createBlob, filterAndListKeys, readBlob, readBlobTo, storedTopoIds, updateBlob, updateLastBlobUpdateTime, validateBlobUpdateTime, validateKey
-
Field Details
-
LOG
public static final org.slf4j.Logger LOG -
aclHandler
-
-
Constructor Details
-
LocalFsBlobStore
public LocalFsBlobStore()
-
-
Method Details
-
prepare
public void prepare(Map<String, Object> conf, String overrideBase, NimbusInfo nimbusInfo, ILeaderElector leaderElector) Description copied from class:BlobStoreAllows us to initialize the blob store. -
startSyncBlobs
Description copied from class:BlobStoreStart the syncing blobs between the local running instance of the BlobStore and others. A no-op for the HdfsBlobStore where HDFS itself does the syncing but for the LocalFsBlobStore ZK state updates are run periodically here.- Overrides:
startSyncBlobsin classBlobStore- Throws:
KeyNotFoundExceptionAuthorizationException
-
createBlob
public AtomicOutputStream createBlob(String key, SettableBlobMeta meta, Subject who) throws AuthorizationException, KeyAlreadyExistsException Description copied from class:BlobStoreCreates the blob.- Specified by:
createBlobin classBlobStore- Parameters:
key- Key for the blobmeta- Metadata which contains the acls informationwho- Is the subject creating the blob- Returns:
- AtomicOutputStream returns a stream into which the data can be written
- Throws:
AuthorizationExceptionKeyAlreadyExistsException
-
updateBlob
public AtomicOutputStream updateBlob(String key, Subject who) throws AuthorizationException, KeyNotFoundException Description copied from class:BlobStoreUpdates the blob data.- Specified by:
updateBlobin classBlobStore- Parameters:
key- Key for the blobwho- Is the subject having the write privilege for the blob- Returns:
- AtomicOutputStream returns a stream into which the data can be written
- Throws:
AuthorizationExceptionKeyNotFoundException
-
getBlobMeta
public ReadableBlobMeta getBlobMeta(String key, Subject who) throws AuthorizationException, KeyNotFoundException Description copied from class:BlobStoreGets the current version of metadata for a blob to be viewed by the user or downloaded by the supervisor.- Specified by:
getBlobMetain classBlobStore- Parameters:
key- Key for the blobwho- Is the subject having the read privilege for the blob- Returns:
- AtomicOutputStream returns a stream into which the data can be written
- Throws:
AuthorizationExceptionKeyNotFoundException
-
setLeaderElector
Sets leader elector (only used by LocalFsBlobStore to help sync blobs between Nimbi.- Specified by:
setLeaderElectorin classBlobStore
-
setBlobMeta
public void setBlobMeta(String key, SettableBlobMeta meta, Subject who) throws AuthorizationException, KeyNotFoundException Description copied from class:BlobStoreSets the metadata with renewed acls for the blob.- Specified by:
setBlobMetain classBlobStore- Parameters:
key- Key for the blobmeta- Metadata which contains the updated acls informationwho- Is the subject having the write privilege for the blob- Throws:
AuthorizationExceptionKeyNotFoundException
-
deleteBlob
Description copied from class:BlobStoreDeletes the blob data and metadata.- Specified by:
deleteBlobin classBlobStore- Parameters:
key- Key for the blobwho- Is the subject having write privilege for the blob- Throws:
AuthorizationExceptionKeyNotFoundException
-
getBlob
public InputStreamWithMeta getBlob(String key, Subject who) throws AuthorizationException, KeyNotFoundException Description copied from class:BlobStoreGets the InputStream to read the blob details.- Specified by:
getBlobin classBlobStore- Parameters:
key- Key for the blobwho- Is the subject having the read privilege for the blob- Returns:
- InputStreamWithMeta has the additional file length and version information
- Throws:
AuthorizationExceptionKeyNotFoundException
-
listKeys
Description copied from class:BlobStoreReturns an iterator with all the list of keys currently available on the blob store. -
shutdown
public void shutdown() -
getBlobReplication
Description copied from class:BlobStoreGets the replication factor of the blob.- Specified by:
getBlobReplicationin classBlobStore- Parameters:
key- Key for the blobwho- Is the subject having the read privilege for the blob- Returns:
- BlobReplication object containing the replication factor for the blob
- Throws:
Exception
-
updateBlobReplication
public int updateBlobReplication(String key, int replication, Subject who) throws AuthorizationException, KeyNotFoundException Description copied from class:BlobStoreModifies the replication factor of the blob.- Specified by:
updateBlobReplicationin classBlobStore- Parameters:
key- Key for the blobreplication- The replication factor the blob has to be setwho- Is the subject having the update privilege for the blob- Returns:
- BlobReplication object containing the updated replication factor for the blob
- Throws:
AuthorizationExceptionKeyNotFoundException
-
checkForBlobOrDownload
- Throws:
KeyNotFoundException
-
checkForBlobUpdate
-
fullCleanup
- Throws:
IOException
-