Marcus Holland Moritz (a Facebook software mainjiniya) akazivisa kuburikidza nechinyorwa mavhezheni ekutanga eDwarFS, kuverenga-chete faira sisitimu yakagadzirirwa kuwedzera kumanikidza uye kudzikisira redundant data.
Iyi faira system inoshandisa iyo FUSE michina uye inomhanya munzvimbo yemushandisi, kodhi yacho yakanyorwa muC ++ uye inogoverwa pasi peGPLv3 rezinesi.
Nezve DwarFS
dwarfs yakafanana nefaira masisitimu seSquashFS, cramfs, uye CromFS mumabasa ako, uye inogona kushandiswa kugadzira mifananidzo mhenyu uye kudzikisira saizi remafaira nenhamba yakakura yezvinyorwa uye duplicate data (semuenzaniso, kuchengetwa kwemifananidzo yemashini chaiwo kana kuunganidzwa kweshanduro dzakasiyana dzezvirongwa).
Panyaya yekumhanya kuwana data, DwarFS iri padanho rakaenzana neiyo squashFS, asi akati wandei pamberi peiyi FS maererano nehunyanzvi hwekumanikidza uye nekumhanyisa kwekufungidzira.
Iyo purojekiti yakagadzirirwa kugadzirisa dambudziko rekugadzirisa kuchengetedza pamwe neshanduro dzakasiyana dzaPerl (munyori weDwarFS anotora chikamu mukuchengetedza iyo CPAN faira).
Pakutanga, takaedza kushandisa Cromfs kumanikidza, Asi zvakatora nguva yakareba kuti vavake mufananidzo uye kugadzikana kwakasiya zvakawanda zvekuda. SquashFS yakashanda zvine mutsindo uye yakashandura mifananidzo ichikurumidza, asi danho rekumanikidza rakanga risingagamuchiriki.
Mazhinji emakodhi eDwarFS yakanyorwa muna 2013. Gore rino, munyori akawana nguva yekuunza kodhi kuruzhinji uye kunyora zvinyorwa. DwarFS inoshandisa iyo Boost neFolly maraibhurari.
Iyo yechando Thrift Facebook bazi raibhurari inoshandiswa kuchengeta metadata. Zvimwe zvinoenderana zvinosanganisira FUSE3 uye iyo lz4, zstd, uye liblzma compression maraibhurari.
DwarFS yakakunda squashFS maererano nekomputa chiyero ka8 nguva, uye maererano nemufananidzo wekusika kumhanya kanokwana kana paunenge uchigadzira mufananidzo unosanganisira 4 kusiyanisa kwakasiyana kwePerl, iko kune mazana maviri nemakumi mana neshanu ePerl.
dwarfs akakwanisa kudzika bhenji kubva pa47GB kuenda pa582MB (1,1% yehukuru hwepakutanga), nepo zvichikonzera squashFS saizi yemufananidzo yaive 4,7 GB. SquashFS yakatora maminetsi makumi matanhatu nemasere kugadzira chifananidzo, nepo DwarFS yakapedza basa iri mumaminitsi gumi nemashanu.
Ose mafaira faira akashandisa iyo ZSTD algorithm yekumanikidza. Uchishandisa LZMA, saizi yemufananidzo weDwarFS yakaderedzwa neimwe 18% (approx. 479MB), asi kumhanya kwekusvika kumufananidzo iwoyo kwakadzikira zvakanyanya.
Miedzo ine data ine mashoma zvakapetwa yakaratidza mukana kwete sekukosha, asi zvakadaro kunoshamisa, kubva kuDwarFS. Semuenzaniso, saizi yemufananidzo wePasibheri Pi OS midzi FS yaive 298MB yeDwarFS uye 364MB yeSquashFS, uye iyo nguva yekuvaka yaive 1 miniti masekondi makumi matatu nemasekondi 36 masekondi 1, zvichiteerana.
Of the akakosha maficha eDwarFS zvinotevera zvinobuda pachena:
- Simba kugona kubvisa redundancy nekumisikidza rakafanana dhata (zvisinei nefaira mafaera) uchishandisa LSH hash mabasa ekuona zvakafanana zvinhu.
- Faira sisitimu block block segmentation kuongorora kudzikisa saizi yeiyo isina kumanikidzwa faira sisitimu uye nekuwedzera kugona kwekushandisa cache cache nekuda kwekuti yakawanda inoda data inopinda.
- Yakawanda-yakamisikidzwa kuitiswa kweiyo imaging utility uye iyo FUSE module, iyo inogona kushandisa ese anowanikwa CPU cores kana uchimhanya.
- Rutsigiro rweyedzo yekugona kubatanidza maLua macontroller ayo anogona kushandiswa kusefa uye kurongedza zvemukati.
- Kudzoreredza modhi iyo inokutendera iwe kuti uchinje iyo yekumanikidza algorithm yemufananidzo watogadzirwa (semuenzaniso, unogona repackage uchishandisa LZMA kana LZ4 pachinzvimbo cheZSTD).
- Mifananidzo inogadzirwa uchishandisa iyo mkdwarfs utility uye yakamisikidzwa uchishandisa iwo madiki mashandisirwo.
Chekupedzisira, kana iwe uchida kuziva zvakawanda nezve iyi faira sisitimu kana uchifarira kugona kuumba kodhi yayo yekodhi, unogona kubvunza iyo ruzivo kana kuwana iyo sosi kodhi Mune inotevera chinongedzo.