efrt-unpack

2019-05-16 admin

efrt-unpack是什么

什么是efrt-unpack,compressed-trie data-structure

efrt-unpack使用教程帮助文档

<div align=“center”>

<div>compression of key-value data</div>

</div>

<div align=“center”>npm install efrt</div>

if your data looks like this:

var data = {
  bedfordshire: 'England',
  aberdeenshire: 'Scotland',
  buckinghamshire: 'England',
  argyllshire: 'Scotland',
  bambridgeshire: 'England',
  cheshire: 'England',
  ayrshire: 'Scotland',
  banffshire: 'Scotland'
};

you can compress it like this:

var str = efrt.pack(data);
//'England:b0che1;ambridge0edford0uckingham0;shire|Scotland:a0banff1;berdeen0rgyll0yr0;shire'

then very! quickly flip it back into:

var obj = efrt.unpack(str);
obj['bedfordshire'];//'England'

Yep,

efrt packs category-type data into a very compressed prefix trie format, so that redundancies in the data are shared, and nothing is repeated.

By doing this clever-stuff ahead-of-time, efrt lets you ship much more data to the client-side, without hassle or overhead.

The whole library is 8kb, the unpack half is barely 2.5kb.

it is based on:

  • <g-emoji class=“g-emoji” alias=“heart_eyes” fallback-src=“https://github.githubassets.com/images/icons/emoji/unicode/1f60d.png”>😍</g-emoji> tamper by the NYTimes
  • <g-emoji class=“g-emoji” alias=“gift_heart” fallback-src=“https://github.githubassets.com/images/icons/emoji/unicode/1f49d.png”>💝</g-emoji> lookups by Mike Koss,
  • <g-emoji class=“g-emoji” alias=“heartbeat” fallback-src=“https://github.githubassets.com/images/icons/emoji/unicode/1f493.png”>💓</g-emoji> bits.js by Steve Hanov

Benchmarks!

Demo!

Basically,
  • get a js object into very compact form
  • reduce filesize/bandwidth a bunch
  • ensure the unpacking time is negligible
  • keep word-lookups on critical-path
var efrt = require('efrt')

var foods = {
  strawberry: 'fruit',
  blueberry: 'fruit',
  blackberry: 'fruit',
  tomato: ['fruit', 'vegetable'],
  cucumber: 'vegetable',
  pepper: 'vegetable'
};
var str = efrt.pack(foods);
//'{"fruit":"bl0straw1tomato;ack0ue0;berry","vegetable":"cucumb0pepp0tomato;er"}'

var obj=efrt.unpack(str)
console.log(obj.tomato)
//['fruit', 'vegetable']

Reserved characters

the keys of the object are normalized. Spaces/unicode are good, but numbers, case-sensitivity, and some punctuation (semicolon, comma, exclamation-mark) are not (yet) supported.

specialChars = new RegExp('[0-9A-Z,;!:|¦]')

efrt is built-for, and used heavily in compromise, to expand the amount of data it can ship onto the client-side. If you find another use for efrt, please drop us a line<g-emoji class=“g-emoji” alias=“balloon” fallback-src=“https://github.githubassets.com/images/icons/emoji/unicode/1f388.png”>🎈</g-emoji>

Performance

efrt is tuned to be very quick to unzip. It is O(1) to lookup. Packing-up the data is the slowest part, which is usually cool.

var compressed = efrt.pack(skateboarders);//1k words (on a macbook)
var trie = efrt.unpack(compressed)
// unpacking-step: 5.1ms

trie.hasOwnProperty('tony hawk')
// cached-lookup: 0.02ms

Size

efrt will pack filesize down as much as possible, depending upon the redundancy of the prefixes/suffixes in the words, and the size of the list.

  • list of countries - 1.5k -> 0.8k (46% compressed)
  • all adverbs in wordnet - 58k -> 24k (58% compressed)
  • all adjectives in wordnet - 265k -> 99k (62% compressed)
  • all nouns in wordnet - 1,775k -> 692k (61% compressed)

but there are some things to consider:

Assuming your data has a low category-to-data ratio, you will hit-breakeven with at about 250 keys. If your data is in the thousands, you can very be confident about saving your users some considerable bandwidth.

Use

IE9+

<script src="https://unpkg.com/efrt@latest/builds/efrt.min.js"></script>
<script>
  var smaller=efrt.pack(['larry','curly','moe'])
  var trie=efrt.unpack(smaller)
  console.log(trie['moe'])
</script>

if you’re doing the second step in the client, you can load just the unpack-half of the library(~3k):

npm install efrt-unpack
<script src="https://unpkg.com/efrt@latest/builds/efrt-unpack.min.js"></script>
<script>
  var trie=unpack(compressedStuff);
  trie.hasOwnProperty('miles davis');
</script>

Thanks to John Resig for his fun trie-compression post on his blog, and Wiktor Jakubczyc for his performance analysis work

MIT

本站文章除注明转载外,均为本站原创或编译。欢迎任何形式的转载,但请务必注明出处。

转载请注明:文章转载自 JavaScript中文网 [https://www.javascriptcn.com]

本文地址:https://www.javascriptcn.com/read-65048.html

文章标题:efrt-unpack

回到顶部