An overview of how Blob works


The Datamaton Content Manager app Blob lets you search, organize, backup and use all your digital content from one place. This is a high-level overview of how it works. You don’t really need to know all this to use it, but its nice to know a little more about an app you might use every day.

Blob is a meta-data indexer

At its heart, Blob is a meta-data indexer – it makes a detailed inventory of all your digital content. Indexing means making an inventory of your digital content and saving it to index files. Meta-data here refers to the fact that Blob only saves relevant information about your content in the index files, not the actual content itself (which stays at its original location). So, just like Google and Bing crawl & index websites, Blob indexes all your content types and storage locations.

So, what metadata does Blob collect & save? The things that you’d normally remember about your content and that you would use to find it later:

  • Emails: Blob collects sender and receiver info, email subject, date, size and email folder name. It also collects emails attributes like read/unread, forwarded, replied etc. and (partial) message text. If the email has attachments, Blob will collect information about them too (the same information it would collect if the file were not an attachment).
  • Text Messages: sender and receiver info, partial message text, date and the channel or group it belongs to. If the text message has attachments, Blob will collect information about them too.
  • Social media posts: poster’s, likers’ and commenters’ names, date, number of likes & comments and attributes. Note however that social media providers now significantly restrict the amount of information they allow apps to collect, so Blob can only index your own posts, comments, likes etc – not that of your friends.
  • Appointments: appointment start & end times, recurrence info, subject, description, calendar location, category etc. when it indexes calendar accounts.
  • Contacts: contact name, email address, phone numbers and attributes. It automatically picks up contact names and email addresses from email messages. Thus, it will auto-complete an email address it saw in your Google mail account even when you send an email from your Microsoft email account.
  • All files: file and folder name, file extension, size, creation & last modification dates and file attributes.
  • Photo, music and video files: album name, title, artist, genre, composer, play duration, camera, GPS information, thumbnails photo and other such standard information (e.g. EXIF data).
  • Cloud files: user tags/comments and file sharing permissions for files stored on web based file storage accounts.
  • Compound files (e.g. ZIP, ISO, Microsoft Outlook PST files etc.): Blob collects information about the parent compound file/email as well as information about the files (or emails) embedded inside them. Thus, a Blob search will also find content that is embedded inside other content (e.g. an email search will find matching emails inside Outlook PST files).

You can now select your content using any of the metadata properties listed above. So, you can search for content using any of these properties; set up backup tasks based on specific values of these properties and organize together content based on any of these content properties (see content selection rules below).

Blob is more than just an indexer

Blob’s objective is to “simplify your digital life”, not just “find your content quickly”. While Blob is an indexer like Google and Bing, it is also much more – it is an aggregator that lets you search, organize, manage and use all your digital content. These capabilities are built on top of the index files Blob creates and maintains.

There’s another core capability that plays a major role – the ability to create arbitrary content selection rules. A Blob content selection rule specifies a) which storage locations it operates on, b) which content type it operates on c) which content property it operates on and d) how it tests the property. Here’s an example: From a) all storage locations b) select emails c) such that email date d) is within the range Jan 1, 2010 and Feb 1, 2010. Another example: From a) C:\, D:\, Yahoo mail account and Google Drive account b) select photos c) such that camera used property d) contains the text iphone. You can combine together multiple rules using “AND” and “OR” operators to create very powerful searches.

Here’s how Blob combines indexing and content selection rules to support different capabilities:

  • Search: A search is just a combination of one or more content selection rules combined with AND or OR operators. When you issue a search, Blob directly searches the index files it created on your hard drive – this is why Blob searches are so fast!
  • Organize: Blob’s Virtual Folders can contain any type of content, stored anywhere. A virtual folder is simply a collection of content selection rules, though you can also drag-drop specific content into it. These rules are dynamically evaluated, so new matching content would automatically be included in it. Since a virtual folder contains only rules and links, it is very light-weight and flexible. You can also assign tags to any type of content, stored anywhere.
  • Manage: Blob lets you move or copy any type of content from anywhere to anywhere. You can do this manually or set up automatic backup tasks. A backup task is actually just a collection of content selection rules coupled with a backup destination and backup frequency. For example, you can create a task to back up “All files and emails that resides in a folder whose name contains the text fin every day at 10am”. Blob schedules backups as separate background processes.
  • Use: You can view all your content from one place and do normal day-to-day operations like read, reply or forward messages; move, copy, download, upload and share files, manage calendar appointments etc. This is obviously much more than what an indexer like Google or Bing would do.

Blob uses your own computer to do its work

Google and Bing use their own servers to index websites and store index files. Blob uses your computer’s CPU, memory and hard drive to index your content and store the index files. Blob’s indexing and backup processes run in the background and at low priority to minimize interference with other work you may be doing on your computer.

This has both advantages and disadvantages. The biggest disadvantage is that how many Data Sources Blob can index and how frequently it can re-index them depends on your own computer’s capabilities. Asking Blob to index terabytes of content across a dozen storage locations might slow it down noticeably, especially if you have an older or less capable computer.

The biggest advantage is that you get complete privacy! Nothing about your content is ever sent to Datamaton’s servers. Blob stores index information as encrypted files on your computer’s hard drive only. You create the password used to encrypt it, and only you can decrypt it. You don’t even have to a create an account with us and don’t have to worry about your content leaking because our servers got hacked.

Blob plays nice with other apps

You don’t have to stop using the apps and tools you currently use to benefit from Blob. For example, when Blob indexes your email accounts (e.g. POP3), it will NOT download messages locally and remove them from the server. You can continue to use your phone or other apps like Microsoft Outlook, Mozilla Thunderbird etc. to access your email. Indexing itself will NOT cause your emails to be marked as read. In fact, Blob will track read/unread, replied and forwarded attributes even when you use other apps to do this. Similarly, you can continue to use your Slack, Facebook, OneDrive etc. apps if you want and use Blob only to search, organize or backup that content.

Blob is a suite of background applications

Blob is actually a collection of many different processes and libraries.

Different components work together to do the overall job:

  • Blob.exe is the user interface (UI) process and the program that you generally interact with. However, this program does not do the actual indexing, backups etc. Other background processes do that, so you can safely close the Blob user interface anytime without affecting anything.
  • A separate library controls interaction with each supported storage location. For example, GMail.dll controls all interaction with Google GMail.
  • Cataloger.exe indexes and periodically re-indexes your storage locations in the background.
  • Refresher.exe looks for dynamical insertion of local devices (CDs, USB drives, phones etc.) so they can be indexed as needed. It also makes sure that periodic re-indexing and backup tasks run when needed.
  • FileMonitor.exe tracks changes to local hard drives so that its index files get updated as soon as local content is created or modified.
  • ImapMonitor.exe tracks your email accounts so new email is immediately indexed.
  • TaskManager.exe runs your backup tasks in the background.
  • SpaceMgr.exe periodically runs in the background to delete temporary or cached files created by Blob.
  • BlobInstallMgr.exe manages license changes and Blob installation and updates.
  • Recrypt.exe lets you set and change the passwords used to encrypt your index files.

Blob starts and schedules these background processes automatically when needed. You need to interact directly only with the Blob.exe user interface.

 

Learn more about Blob, download the always-free version, download the 45-day trial version, or buy it. Please Contact Us if you have any questions.

Leave a comment

Your email address will not be published.