Most models I've seen have at most a few hundred MB of parameters, have complex connections that can only be modeled as a set of many different 1 or 2D arrays with different sizes. Further, this DB stresses that it handles sparse sparse data, any most ML data is not sparse.
It doesn't seem like the best application. On one of their pages it mentions they ingest BAM records, which is for biological sequences. I'm guessing some DNA storage applications.
This brings up a question: in what fields does one find heavy use of large, sparse matrices that need to be persisted and queryable?
In my mind, sparse matrices typically occur in the context of graphs/relationships, e.g. PageRank, logistic networks, adjacency matrices, etc. They also tend to be a property of Hessian matrices (2nd order derivative for a multivariate system). But typically these are intermediate quantities that are discarded after a computation completes.
Jake from TileDB, Inc. Genomics is a big field where sparse matrix storage is needed. Human genomes are stored as a diff off of a reference, which as you indicated forms a graph which can be represented as a sparse matrix. In other fields of genomics, such as metagenomics, fragments of DNA when analyzed also have a graph like structure.
TileDB supports both dense and sparse arrays. It was designed around the concept of handling sparse arrays but dense arrays can be thought of a degenerate case of sparse array storage in TileDB. For dense arrays tile extents are contiguous and we don't materialize the coordinate values. This way all the concepts are the same and we can capture both use-cases. Sparse annotations to dense array values, such as NA or Null handling can also be captured as a sparse array fragment layered over a backing dense array.
I agree with you that for most use cases, storage will be dense. But it is useful to have one system that can handle both representations efficiently, with the sparse case not added on as an afterthought (it also makes the system simpler).
Stavros form TileDB, Inc. here: Another area of great interest to us is point cloud data, which are essentially 3D points in a super sparse space. In fact, any application dealing with spatio-temporal data (which are 2D/3D/4D data that tend to be sparse and skewed) can take advantage of TileDB.
As an ML guy I can say that sparse is pretty common in ML. Text data, market basket data, graph Laplacians, adjacency matrix, sequence fragment data are all rich in sparse matrix computation and storage operations. For a moment I thought the "not" was a typo. Lack of support for sparse matrices often becomes a serious inconvenience in a tool, very happy that TileDB folks have given thought to the sparsity requirement from get-go
