Ciencias NaturalesInglésHugo

Donny Winston

Donny Winston
Made as simple as possible, but not simpler.
Página de inicioFeed AtomMastodon
language
Ciencias NaturalesInglés
Publicado

A lot of extract-transform-load (ETL) work requires unloading and un-transforming first. Rather than \ETL\, it’s \L_A^{-1}T_A^{-1}ET_BL_B\. What the data provider did is \A\. What you want to do is \B\. The data provider gave you a “dump” of their data. You don’t know what it means. If you did, you could extract (subset) from it according to your needs – filter entities by some meaningful criteria and collect selected attributes.

Ciencias NaturalesInglés
Publicado

Wikidata uses opaque identifiers for its catalogued information resources. For example, the statement “wd:Q42 wdt:P69 wd:Q691283” may map to the label sequence “‘Douglas Adams’ ‘educated at’ ‘St John’s College’” with a language-locale preference of English-US. Opaque naming is wise for internationalization.

Ciencias NaturalesInglés
Publicado

Imagine a data system modeled as three parts: an interface, a processor, and a repository. The repository “contains” information. The processor receives symbol streams to alter or retrieve information from the repository, and the processor outputs symbol streams. The interface is the medium, the opaque surface, of symbol-stream exchange between you and the processor. 1 What information is “in” the repository?

Ciencias NaturalesInglés
Publicado

After my last note on identifiers, Leo Talirz pointed me to a great riff 1 on Tim Berners-Lee’s classic note 2 on “cool URIs”. In the “Cool DOIs” article, Fenner breaks down a DOI into three parts: proxy, prefix, and suffix. A proxy is a server that maintains a map from prefixes to registrants. Example proxies are https://doi.org/ and https://hdl.handle.net/. An example prefix is 10.5281.

Ciencias NaturalesInglés
Publicado

Data protocols vary over project lifetimes, and many projects involve parameter sweeps. You might see filesystem directory structures evolve naming schemes like the following 1 : # let's not overthink this at first. concentration_A_0.25/ # hierarchy is good, right? concentration_A/0.25/ # paths are getting too long. conc_A/0.25/ # change to percentages. clever!

Ciencias NaturalesInglés
Publicado

Approaches to data citation may span classes of big-O complexity, for both space (memory/storage) and time (compute/transfer). Dataset revisions may be minted and persisted without any delta encoding / structural sharing. The main mechanism of reproduction for citations in this case is restoration. Space complexity is high, as storage needs are high.

Ciencias NaturalesInglés
Publicado

Against what bases are queries against your data evaluated? If you only expose a single “data base” that changes over time, then data citations cannot be a combination of query and basis. When citing a passage in a book, the edition/variation of its publication, i.e. the thing that is assigned an ISBN, is the basis. Optionally, a citation may include a “query” against this basis – a page number, page range, chapter number/title, etc.

Ciencias NaturalesInglés
Publicado

Many tests use oracles , where you know the answers for some inputs and you check those correspondences. To cover more of the input state space, you can generate random inputs and check some properties for each corresponding output. You don’t have an enumeration of exact answers like with oracles, but you can check things like the output always being greater than zero, etc.

Ciencias NaturalesInglés
Publicado

In a collaboration, data objects are produced at many sites. To make the data objects findable, you may steward a central, searchable index for their metadata. How then do you make the data objects accessible for download? One common solution is to centralize the custodianship – have all sites upload copies of their data objects to a central store. The central store may partition storage across several physical servers behind the scenes (e.g.