Storing and accessing binary data (BLOBs)
:: services/blobbox module

BLOB stands for a Binary Large OBject. It can be anything you like. You can store any piece of binary data on USOS API servers (provided, of course, we give you a proper administrative Consumer Key).

This module is primarily intended to be used by our own applications. You can use it when (for some reason) you don't have access to any other secure storage space (access to USOS API is unlimited from any location). Or, use it simply because you don't want to administer big data files by yourself (its easier if we do that in one place). Also, some other USOS API functionality (like for example the fileshare module) is built on top of the blobbox module.

We provide you with (possibly unlimited) storage space. What we ask in return is that you keep track of your blob IDs. You should delete the blobs you don't need anymore (or give them a decent expiry time). If you allow your users to store files here, keep track of how much they store (don't let them do "too much"). Also, remember to transmit your secret data over HTTPS (as we don't enforce that).

By default, blobs are accessible for you only, but you can make them readable by others too (see also the fileshare module).

Note: If you use an Access Token in your request, USOS API will remember the reference between a given user and the blob (i.e. you will be allowed to see your stats filtered by specific user). We encourage you to store such references. In future, we may use this data to find the most "storage greedy" users and poster them about it. However, keep in mind, that currently these references do not actually do anything (in terms of functionality).

Methods

blob Get information on a given blob. (administrative)
blobs Get information on given blobs. (administrative)
chmod Update an existing blob's sharing options. (administrative)
consumer Get consumer's storage usage statistics. (administrative)
delete Delete a blob (or multiple blobs). (administrative)
get Get the contents of a blob.
inspect_blob BETA Inspect the blob's contents and retrieve basic metadata
put Put a blob into the database, using form-data encoding. (administrative)
put_base64 Put a blob into the database, using base64 encoding. (administrative)
put_sha1 Put a blob into the database, using a SHA-1 hash. (administrative)
put_text Put a text blob into the database. (administrative)
put_unsigned Upload data using form-data encoding.
user Get storage usage statistics for a given user. (administrative)
users Get storage usage statistics for given users. (administrative)
users_all Get storage usage statistics for all users. (administrative)

services/blobbox/blob

Administrative: This method requires a proper Administrative Consumer Key. Contact us to get one.

Consumer: required (administrative) Token: ignored Scopes: n/a SSL: not required
https://apps.usos.pw.edu.pl/services/blobbox/blob
Get information on a given blob.
blob_id required

Blob ID. You may get information only on blobs which were uploaded by yourself (using your consumer key). All foreign blobs will appear as inexistent.

If you want to get the content of the blob, use the get method.
fields optional

Default value: (all)

Pipe-separated list of informational fields/sections you're interested in. This must be any subset of keys, which are described in the returns section.
format optional

Default value: json

Format in which to return values. See supported output formats.
callback optional Required only if you've chosen jsonp as a return format.
Plus required standard OAuth Consumer signing arguments: oauth_consumer_key, oauth_nonce, oauth_timestamp, oauth_signature, oauth_signature_method, oauth_version. Token is not required.

Returned value:

A dictionary of selected fields and their values.

Available fields:

  • size - size of the blob, in bytes,
  • user_id - user ID whose Access Token was used when uploading the blob, or null when the blob has not been bound to any user,
  • read_access - as described in the put_text method,
  • date_created - datetime string, the time when the blob was first created,
  • date_last_read - datetime string, the time when the blob was last read (a get method was used on it), or null when it hasn't been read yet,
  • times_read - total number of times the blob was read (a get method was used on it),
  • expires_after - datetime string, the time after which the blob might get automatically deleted (it is not safe to access it afterwards), or null when blob in not meant to be automatically deleted (it can still be deleted manually).
  • expires_before - datetime string, the time after which the blob will be deleted, no matter what (unless the expiry dates get changed), or null when the expiry date was not set.

If the blob does not exist (or you don't own it), you will get a HTTP 400 error.

services/blobbox/blobs

Administrative: This method requires a proper Administrative Consumer Key. Contact us to get one.

Consumer: required (administrative) Token: ignored Scopes: n/a SSL: not required
https://apps.usos.pw.edu.pl/services/blobbox/blobs
Get information on given blobs.
blob_ids required

Pipe-separated list of blob IDs (no more than 500).

You may get information only on blobs which were uploaded by yourself (using your consumer key). All foreign blobs will appear as inexistent.

If you want to get the content of the blob, use the get method.
fields required Pipe-separated list of informational fields/sections you're interested in. This must be any subset of keys, which are described in the returns section of the blob method.
format optional

Default value: json

Format in which to return values. See supported output formats.
callback optional Required only if you've chosen jsonp as a return format.
Plus required standard OAuth Consumer signing arguments: oauth_consumer_key, oauth_nonce, oauth_timestamp, oauth_signature, oauth_signature_method, oauth_version. Token is not required.

Returned value:

A dictionary. Blob IDs will be mapped to keys of this dictionary. Each value will be a dictionary of the same structure as described in the blob method.

For all invalid references (blob IDs which point to inexistent blobs, or blobs you do not own), the value will be null (instead of a dictionary).

services/blobbox/chmod

Administrative: This method requires a proper Administrative Consumer Key. Contact us to get one.

Consumer: required (administrative) Token: ignored Scopes: n/a SSL: not required
https://apps.usos.pw.edu.pl/services/blobbox/chmod
Update an existing blob's sharing options.
blob_id required ID of the blob (you got it from the put_* method). It has to exist and you have to be its owner.
read_access optional

Default value: skip

One of the options listed in the put_text method, or a special skip option to indicate that you don't want to change the previous settings.
min_age optional

Default value: skip

One of the options listed in the put_text method, or a special skip option to indicate that you don't want to change the previous settings.

Just to clear up: We will count this age starting from the current time (and not from the original upload time).
max_age optional

Default value: skip

One of the options listed in the put_text method, or a special skip option to indicate that you don't want to change the previous settings.

Just to clear up: We will count this age starting from the current time (and not from the original upload time).
format optional

Default value: json

Format in which to return values. See supported output formats.
callback optional Required only if you've chosen jsonp as a return format.
Plus required standard OAuth Consumer signing arguments: oauth_consumer_key, oauth_nonce, oauth_timestamp, oauth_signature, oauth_signature_method, oauth_version. Token is not required.

Returned value:

Currently, this method always returns the same value: a dictionary with single success key with values of true.

services/blobbox/consumer

Administrative: This method requires a proper Administrative Consumer Key. Contact us to get one.

Consumer: required (administrative) Token: ignored Scopes: n/a SSL: not required
https://apps.usos.pw.edu.pl/services/blobbox/consumer
Get consumer's storage usage statistics.
fields optional

Default value: hard_count|hard_vsize

Pipe-separated list of informational fields/sections you're interested in. This must be any subset of keys, which are described in the returns section.
format optional

Default value: json

Format in which to return values. See supported output formats.
callback optional Required only if you've chosen jsonp as a return format.
Plus required standard OAuth Consumer signing arguments: oauth_consumer_key, oauth_nonce, oauth_timestamp, oauth_signature, oauth_signature_method, oauth_version. Token is not required.

Returned value:

A dictionary of selected fields and their values.

Available fields:

  • all_count - total number of blobs currently held under your ownership,
  • hard_count - number of your blobs which, based on your expiry preferences, cannot be deleted right now,
  • soft_count - number of your blobs which have passed the limit set upon them with the min_age parameter, but have not yet been deleted (these blobs may be deleted at any moment),
  • all_vsize - total size of all your blobs (in bytes),
  • hard_vsize - total size of all your hard blobs,
  • soft_vsize - total size of all your soft blobs.

  • all_blob_ids - the list of all your blobs (their IDs).

    Note: You should store all needed blob IDs locally (in your own database). These fields (*_blob_ids) are intended to be used for integrity checks only. The output can be huge at times!

  • hard_blob_ids - the list of all your hard blobs.
  • soft_blob_ids - the list of all your soft blobs,

vsize stands for virtual size - the sum of lengths of your blobs. The amount of physical storage which is actually used for your data may vary, depending on the underlying implementation and system settings.

I.e. if you upload the same blob 100 times, each weighting 1 MB, you will get 100 blob_ids and your all_vsize will increase by 100 MB. However, internally the file will be stored only once. Similarly, we may also compress your blobs before storing, etc.

services/blobbox/delete

Administrative: This method requires a proper Administrative Consumer Key. Contact us to get one.

Consumer: required (administrative) Token: ignored Scopes: n/a SSL: not required
https://apps.usos.pw.edu.pl/services/blobbox/delete
Delete a blob (or multiple blobs).
blob_ids required Pipe-separated list of blob IDs (no more than 500). You may only delete blobs which you have uploaded yourself. All other blob IDs will be ignored.
format optional

Default value: json

Format in which to return values. See supported output formats.
callback optional Required only if you've chosen jsonp as a return format.
Plus required standard OAuth Consumer signing arguments: oauth_consumer_key, oauth_nonce, oauth_timestamp, oauth_signature, oauth_signature_method, oauth_version. Token is not required.

Returned value:

A dictionary of the following structure:

  • matched - list of blob IDs which were actually deleted (they still existed and were owned by you).

    Note, that if you set expiry times for your blobs, then they might have been deleted automatically before you called this method.

services/blobbox/get

Consumer: required Token: ignored Scopes: n/a SSL: not required
https://apps.usos.pw.edu.pl/services/blobbox/get
Get the contents of a blob.
blob_id required Blob ID. You may only read blobs which you have access to (see the read_access parameter in the put_text method). Unaccessible blobs will appear as inexistent.
Plus required standard OAuth Consumer signing arguments: oauth_consumer_key, oauth_nonce, oauth_timestamp, oauth_signature, oauth_signature_method, oauth_version. Token is not required.

Returned value:

HTTP 200 response with raw contents of the blob.

If the blob does not exist (or you do not have access to it), we will respond with a HTTP 400 error.

services/blobbox/inspect_blob

Consumer: optional Token: ignored Scopes: n/a SSL: not required
https://apps.usos.pw.edu.pl/services/blobbox/inspect_blob

This is a BETA method. We're looking for beta-testers. Until we find them, there's a substantial probability it won't stay backwards-compatible! If you are planning on using this method, please let us know. Then, we will work with you and move it out of beta as soon as we can.

Inspect the blob's contents and retrieve basic metadata.

This method attempts to inspect the binary contents of the blob and extract some basic metadata (without the need for you to actually download the blob itself).

Keep in mind however, that this method is kept primarilly for debugging purposes: it runs only the very simplest of tests, and the accuracy of these tests is not guaranteed. It is your responsibility to store (on your servers) detailed metadata on your blobs, if you really need such data.
blob_id required

Blob ID. You can only inspect blobs which you have access to (see the read_access parameter in the put_text method). Unaccessible blobs will appear as inexistent.
format optional

Default value: json

Format in which to return values. See supported output formats.
callback optional

Required only if you've chosen jsonp as a return format.
Plus optional standard OAuth Consumer signing arguments: oauth_consumer_key, oauth_nonce, oauth_timestamp, oauth_signature, oauth_signature_method, oauth_version. Token is not required.

Returned value:

A dictionary of following fields and their values:

  • content_type - the content-type of the blob.

    This content type is based on running some basic heuristics, and not on actual knowledge. It will sometimes return invalid results.

services/blobbox/put

Administrative: This method requires a proper Administrative Consumer Key. Contact us to get one.

Consumer: required (administrative) Token: optional Scopes: n/a SSL: required
https://apps.usos.pw.edu.pl/services/blobbox/put

Put a blob into the database. Use form-data encoding to pass the binary data.

Please note that we require SSL and OAuth PLAINTEXT signature for this method to work (regular OAuth 1.0 SHA1 signature doesn't work correctly with multipart/form-data encoding).

Please include an Access Token if you have one (why?)
data required The file to be uploaded. Your request has to use multipart/form-data encoding and the data key must have a "filename" in its Content-Disposition header.
read_access optional

Default value: consumer

Same as in the put_text method.
min_age optional

Default value: none

Same as in the put_text method.
max_age optional

Default value: none

Same as in the put_text method.
format optional

Default value: json

Format in which to return values. See supported output formats.
callback optional Required only if you've chosen jsonp as a return format.
Plus required standard OAuth Consumer signing arguments: oauth_consumer_key, oauth_nonce, oauth_timestamp, oauth_signature, oauth_signature_method, oauth_version. Plus optional oauth_token for Token authorization.

Returned value:

Same as in the put_text method.

services/blobbox/put_base64

Administrative: This method requires a proper Administrative Consumer Key. Contact us to get one.

Consumer: required (administrative) Token: optional Scopes: n/a SSL: not required
https://apps.usos.pw.edu.pl/services/blobbox/put_base64

Put a blob into the database. Use base64 encoding to pass the binary data.

Please include an Access Token if you have one (why?)
data required

Base64-encoded data to be stored (no more than 1 MB).

This method is not recommended. It is more efficient to use the put method (which also allows you to store bigger blobs than 1MB!).
read_access optional

Default value: consumer

Same as in the put_text method.
min_age optional

Default value: none

Same as in the put_text method.
max_age optional

Default value: none

Same as in the put_text method.
format optional

Default value: json

Format in which to return values. See supported output formats.
callback optional Required only if you've chosen jsonp as a return format.
Plus required standard OAuth Consumer signing arguments: oauth_consumer_key, oauth_nonce, oauth_timestamp, oauth_signature, oauth_signature_method, oauth_version. Plus optional oauth_token for Token authorization.

Returned value:

Same as in the put_text method.

services/blobbox/put_sha1

Administrative: This method requires a proper Administrative Consumer Key. Contact us to get one.

Consumer: required (administrative) Token: optional Scopes: n/a SSL: not required
https://apps.usos.pw.edu.pl/services/blobbox/put_sha1

Put a text blob into the database, using a SHA-1 hash.

Usually you will use this method after you upload a file with put_unsigned. However, you may also attempt to use this method if you have a reason to suspect that your content may already exist (or it existed recently) in our blobbox database. USOS API will try to find the content and if it succeeds, it will be linked to a new blob ID.

Please include an Access Token if you have one (why?)
sha1 required SHA-1 hash of the data to be stored.
datasize required Length of the data, in bytes (for some extra security).
read_access optional

Default value: consumer

Same as in the put_text method.
min_age optional

Default value: none

Same as in the put_text method.
max_age optional

Default value: none

Same as in the put_text method.
format optional

Default value: json

Format in which to return values. See supported output formats.
callback optional Required only if you've chosen jsonp as a return format.
Plus required standard OAuth Consumer signing arguments: oauth_consumer_key, oauth_nonce, oauth_timestamp, oauth_signature, oauth_signature_method, oauth_version. Plus optional oauth_token for Token authorization.

Returned value:

Same as in the put_text method.

services/blobbox/put_text

Administrative: This method requires a proper Administrative Consumer Key. Contact us to get one.

Consumer: required (administrative) Token: optional Scopes: n/a SSL: not required
https://apps.usos.pw.edu.pl/services/blobbox/put_text

Put a text blob into the database.

Please include an Access Token if you have one (why?)
data required

Data to be stored (no more than 1 MB).

This method can be used for text blobs only. You'll get an error if you try to send binary data which cannot be read as utf8-encoded text. Use put or put_base64 methods if you're not sure if your content is binary or not.

If 1 MB id not enough, use put or put_unsigned methods to upload your data.
read_access optional

Default value: consumer

Sharing options. One of the following values:

  • public - any Consumer (even non-administrative one) in possession of the blob_id will be allowed to read the data. You will also be able to use the fileshare module to share the blob publicly with end users (they won't need to use any Consumer Key).
  • consumer - only you (the Consumer) will be allowed to read the data.
  • More sharing options may come on demand. E.g. users option, which would allow to share a blob only with the chosen USOS users, or consumers options which would do the same for chosen Consumer Keys. Contact us if you need one.

Regardless of what you choose, only you (the Consumer) will be allowed to update or delete the data. You can use the sharing options to allow others to read your data, but you cannot allow others to change it in any way.
min_age optional

Default value: none

By default (if min_age and max_age are set to none), your data is kept indefinitely (or, until explicit removal). When passing this parameter, you allow us to delete your data after the specified time.

Integer, or a special none value. The minimum number of seconds to store the data (counting from now). We will not delete your data before this time, but we may choose to store it longer, based, for example, on its size and last access time (blobs which are regularly read may be kept much longer, even indefinitely).

This parameter is useful when you want to allow public read access to a big chunk of temporary data (like an email attachment), but you're not sure how long it takes until people will actually stop accessing it. You may combine this parameter with max_age or expires.
max_age optional

Default value: none

By default (if min_age and max_age are set to none), your data is kept indefinitely (or, until explicit removal). By passing this parameter, you tell us to delete it after a specified time.

Integer, or a special none value. The maximum time (in seconds) after which the data must be deleted. Note, that the data may be deleted sooner, if you use min_age parameter.
format optional

Default value: json

Format in which to return values. See supported output formats.
callback optional Required only if you've chosen jsonp as a return format.
Plus required standard OAuth Consumer signing arguments: oauth_consumer_key, oauth_nonce, oauth_timestamp, oauth_signature, oauth_signature_method, oauth_version. Plus optional oauth_token for Token authorization.

Returned value:

A dictionary of the following structure:

  • blob_id - newly created blob ID, which you can use to reference this blob.

In case of storage error (i.e. out of space), we will respond with HTTP 500 status and USOS API administrators will be notified.

services/blobbox/put_unsigned

Consumer: ignored Token: ignored Scopes: n/a SSL: not required
https://apps.usos.pw.edu.pl/services/blobbox/put_unsigned

This method does not put the blob into the database. It does however temporarily save the file and wait for you to complete the put operation. In order for such data to become a "fully qualified" blob, you need to call the put_sha1 method afterwards.

This method does not require OAuth signatures, so you could use it to upload files directly from your user's browser. However you would also need to find a way to retrieve SHA1 of the uploaded file from your user's browser which is tricky... Probably it would be easier for you to simply store the file temporarily on your server, and then use the put to store it.

Important: You have to call the put_sha1 method shortly after finishing the file upload. Otherwise, your file will be deleted.
data required The file to be uploaded. Your request has to use multipart/form-data encoding and the data key must have a "filename" in its Content-Disposition header.
format optional

Default value: json

Format in which to return values. See supported output formats.
callback optional Required only if you've chosen jsonp as a return format.
No additional OAuth arguments are required. If you provide any, they will be ignored.

Returned value:

Currently, this method always returns the same value: a dictionary with single success key with the value of true.

services/blobbox/user

Administrative: This method requires a proper Administrative Consumer Key. Contact us to get one.

Consumer: required (administrative) Token: ignored Scopes: n/a SSL: not required
https://apps.usos.pw.edu.pl/services/blobbox/user

Get storage usage statistics for a given user. Note, that user stats do not include blobs uploaded by other consumers! You have access only to stats related to your application.
user_id required ID of the user. If you provide an invalid user ID, you will get a HTTP 400 error.
fields optional

Default value: hard_count|hard_vsize

Pipe-separated list of fields (as in the consumer method).
format optional

Default value: json

Format in which to return values. See supported output formats.
callback optional Required only if you've chosen jsonp as a return format.
Plus required standard OAuth Consumer signing arguments: oauth_consumer_key, oauth_nonce, oauth_timestamp, oauth_signature, oauth_signature_method, oauth_version. Token is not required.

Returned value:

A dictionary of the same structure as the one returned in the consumer method. However, all the values will be computed against blobs uploaded by matched users only.

services/blobbox/users

Administrative: This method requires a proper Administrative Consumer Key. Contact us to get one.

Consumer: required (administrative) Token: ignored Scopes: n/a SSL: not required
https://apps.usos.pw.edu.pl/services/blobbox/users

Get storage usage statistics grouped by users. Note, that user stats do not include blobs uploaded by other consumers! You have access only to stats related to your application.
user_ids required Pipe-separated list of user IDs (no more than 500).
fields optional

Default value: hard_count|hard_vsize

Pipe-separated list of fields (as in the consumer method).
format optional

Default value: json

Format in which to return values. See supported output formats.
callback optional Required only if you've chosen jsonp as a return format.
Plus required standard OAuth Consumer signing arguments: oauth_consumer_key, oauth_nonce, oauth_timestamp, oauth_signature, oauth_signature_method, oauth_version. Token is not required.

Returned value:

A dictionary. User IDs will be mapped to keys of this dictionary. Each value will be a dictionary of the same structure as the one returned in the consumer method. However, all the values will be computed against blobs uploaded by matched users only.

For all invalid references (unknown user IDs), the value will be null.

services/blobbox/users_all

Administrative: This method requires a proper Administrative Consumer Key. Contact us to get one.

Consumer: required (administrative) Token: ignored Scopes: n/a SSL: not required
https://apps.usos.pw.edu.pl/services/blobbox/users_all

This method works like the users method, but it returns all users in whose name you store any data. Note, that the result might be very large.
fields optional

Default value: hard_count|hard_vsize

Pipe-separated list of fields (as in the consumer method).
format optional

Default value: json

Format in which to return values. See supported output formats.
callback optional Required only if you've chosen jsonp as a return format.
Plus required standard OAuth Consumer signing arguments: oauth_consumer_key, oauth_nonce, oauth_timestamp, oauth_signature, oauth_signature_method, oauth_version. Token is not required.

Returned value:

Same as the the users method.

Note: Only users with all_count greater than zero will be included.
USOS API ver. 7.1.0.0-4, b58e51a8, dirty (2024-09-03)