Commit Graph

922 Commits

Author SHA1 Message Date
Joonas Koivunen
8c10a0ad0d fix: remove calling ulimit
while this may have succeeded on github actions it probably isn't wise
to run it every time at least automatically.
2020-08-07 21:05:38 +03:00
Joonas Koivunen
e38c1ff04c refactor: try splitting the huge func 2020-08-07 15:06:55 +03:00
Joonas Koivunen
fdc5f8aa84 refactor: make PostOrderIterator 'static
kind of forgot the String and Vec<u8> while trying to impl the Iterator
trait to return non-static values.
2020-08-07 15:06:55 +03:00
Joonas Koivunen
41566e4759 refactor(postorderiter): remove redundant visibilities 2020-08-07 15:06:55 +03:00
Joonas Koivunen
684ecdc9d3 doc: note lld-9 ulimit -s bisection 2020-08-07 15:06:55 +03:00
Joonas Koivunen
f24f9bda42 refactor: reorder items, fix doc 2020-08-07 15:06:55 +03:00
Joonas Koivunen
761124884f feat: block size limit 2020-08-07 15:06:55 +03:00
Joonas Koivunen
b134593708 doc: minor too short sentence 2020-08-07 15:06:55 +03:00
Joonas Koivunen
5d8b124367 doc: suggestions from code review
Thanks to ljedrz for the many many fixes.

Co-authored-by: ljedrz <ljedrz@users.noreply.github.com>
2020-08-07 15:06:55 +03:00
Joonas Koivunen
7d7915a9da doc: remove mention of wrap in => wrap with 2020-08-07 15:06:55 +03:00
Joonas Koivunen
7bbff65bdd fix: DirBuilder::is_empty 2020-08-07 15:06:55 +03:00
Joonas Koivunen
39583a77f1 refactor: remove extra initialization 2020-08-07 15:06:55 +03:00
Joonas Koivunen
1c71e3c08d refactor: tests repeated the verify with block dump fn 2020-08-07 15:06:55 +03:00
Joonas Koivunen
f3b92d48d6 fix: replace todo! with Err(PathEndsInSlash)
this can leave empty nodes in the tree.
2020-08-07 15:06:55 +03:00
Joonas Koivunen
9e948f943f perf: render Cids to bytes without allocating a Vec
not sure how much sense does this make, given that the byte writing fns
might be less inlined. would be nice to have some benchmarks.

this also adds a test to verify directory cid when using cidv1 even
though it's quite unclear to me if those are allowed in unixfs dirs.
2020-08-07 15:06:55 +03:00
Joonas Koivunen
3e499e161e perf: custom dag-pb serialization
use the BTreeMap<String, Leaf> directly to output the PBLink alike
bytes without going through the mapping into Vec. Does not support Cid
v1 though yet.
2020-08-07 15:06:55 +03:00
Joonas Koivunen
653d079cc9 doc: add note on HAMT sharding, adjust 2020-08-07 15:06:55 +03:00
Joonas Koivunen
a90826b48d perf: avoid clearing block_buffer
this should lead to less zero extending but it's questionable if this is
more performant.
2020-08-07 15:06:55 +03:00
Joonas Koivunen
dc2c0217b9 feat: /add?progress=true
adds the progress reporting, which is a progress notification to the
response stream on every block write *in addition to* a notification
*after* the file has been read (order with the `Added` message doesn't
seem to have been specified).
2020-08-07 15:06:55 +03:00
Joonas Koivunen
5534fef2b8 refactor: wrap_in_directory => wrap_with_directory
just aligning the terminology and trying not to invent new terms.
2020-08-07 15:06:55 +03:00
Joonas Koivunen
14112c3dca feat: /add?wrap_with_directory=true
previous implementation included "defaulting the name for the wrapper
dir" which was extra and not expected by the interface tests.
2020-08-07 15:06:55 +03:00
Joonas Koivunen
65acedae1d fix: normalize prefix slash away to enable more tests 2020-08-07 15:06:55 +03:00
Joonas Koivunen
4077de1a0f fix: report adderrors so that js-ipfs-http-client errors
the mechanism is to just push a new line of MessageResponse. it's a
hack, but at least the js-ipfs-http-client fails.
2020-08-07 15:06:55 +03:00
Joonas Koivunen
8319e8bd97 chore: remove extra clone (clippy) 2020-08-07 15:06:55 +03:00
Joonas Koivunen
dd8d9d9bfe conformance: enable more directory tests 2020-08-07 15:06:55 +03:00
Joonas Koivunen
57737b49c6 refactor: detupleify into (Owned)?TreeNode 2020-08-07 15:06:55 +03:00
Joonas Koivunen
a5b52b3682 doc: add initial documentation 2020-08-07 15:06:55 +03:00
Joonas Koivunen
6ba4195500 refactor: split dir/builder.rs up
* split off dir_builder.rs
* split off PostOrderIterator
* split bufferedtreebuilder off
2020-08-07 15:06:55 +03:00
Joonas Koivunen
548dbe2873 chore: appease clippy
single while let Some(_) = ... changed to is_some().
2020-08-07 15:06:55 +03:00
Joonas Koivunen
bb9555b14e doc: explain anyhow dependency 2020-08-07 15:06:55 +03:00
Joonas Koivunen
f0bb5b41fd feat: create empty directories without metadata
this is the go-ipfs 0.5 level support for add and directories. next up
will be parsing the headers as unixfsv1.5 metadata and using those with
the directories *and* files.
2020-08-07 15:06:55 +03:00
Joonas Koivunen
ccd6bbe248 feat: build directory trees on /add 2020-08-07 15:06:53 +03:00
Joonas Koivunen
c59a4cbe22 refactor: try make /add streaming 2020-08-07 15:05:05 +03:00
Joonas Koivunen
31e2a5ee5a fix: remove unneeded mut in render_directory 2020-08-07 15:05:05 +03:00
Joonas Koivunen
de9247574d feat: generate actual dag-pb 2020-08-07 15:05:05 +03:00
Joonas Koivunen
fa16bfd397 feat: unixfs tree builder 2020-08-07 15:05:05 +03:00
bors[bot]
926e2f6347
Merge #292
292: Use tokio::test in http instead of its async_std counterpart r=koivunej a=ljedrz

As `tokio` is the async executor in `http`.

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-07 08:40:22 +00:00
ljedrz
d2016f0193 fix: use tokio::test in http instead of its async_std counterpart
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-07 09:27:15 +02:00
bors[bot]
70bccd8873
Merge #280
280: Patch a Bitswap leak r=koivunej a=ljedrz

We currently don't properly clean up after a connection to a `Bitswap` peer is closed, which leads to leaking `Ledger`s; funnily enough, it seems that just uncommenting a pre-existing line solves the issue. In addition, add a relevant test.

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-06 15:08:27 +00:00
bors[bot]
0adaf255c6
Merge #291
291: fix: add timeouts to connect_two tests r=ljedrz a=koivunej

gha windows was stuck on this. it's not nice since we cannot see any
testing output before the run is cancelled.

Co-authored-by: Joonas Koivunen <joonas@equilibrium.co>
2020-08-06 14:24:05 +00:00
Joonas Koivunen
3e2554aa47 fix: add timeouts to connect_two tests
gha windows was stuck on this. it's not nice since we cannot see any
testing output before the run is cancelled.
2020-08-06 17:20:47 +03:00
ljedrz
d20131077a fix: consider the possibility that a peer is no longer available in send_block
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-06 16:02:05 +02:00
ljedrz
b5c0dcf907 docs: comment on not dropping peer stats on disconnects
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-06 16:02:05 +02:00
ljedrz
576ee6b92e fix: tweak a comment (drive-by)
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-06 16:02:05 +02:00
ljedrz
2dfabdf66a fix: patch a Bitswap leak and test it
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-06 16:02:04 +02:00
bors[bot]
0fb84c0b49
Merge #281
281: Some Kademlia debugging r=koivunej a=ljedrz

These changes were sparked by the following 2 observations:
- the logs sometimes indicate that a Kademlia query was executed twice
- `finish_subscription` didn't always result in futures being awoken

While the former remains a mystery (its occurrence is not correlated with subscriptions, meaning it's either some polling issue that eludes me or a bug in `libp2p`), investigating the `kad`<>`Subscription` route yielded a few improvements, namely:
- tweaked `kad` log levels
- proper `SubscriptionFuture` handling in `put_block`
- improved `SubscriptionRegistry` logs and a small fix
- simpler `SubscriptionFuture` type handling (it always returned a `Result`, so now it's a default)
- a `debug_assert` checking that we don't trigger zero-wake cases in `finish_subscription` during tests
- improved `swarm_api` test (that sometimes caused issues with the new `debug_assert`, but could be improved regardless)

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-06 13:39:18 +00:00
bors[bot]
b99288b7b1
Merge #290
290: Some maintenance r=koivunej a=ljedrz

Some maintenance work so as not to conflict too much with the pending PRs:
- depend on cid instead of libipld where only Cid is used
- update the `domain` dep (big `Cargo.lock` wins)

cc #75 

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-06 13:23:45 +00:00
bors[bot]
47d2674d57
Merge #288
288: fix: update async-stream to 0.3 r=ljedrz a=koivunej

this uses the rust 1.45 features to remove recursion inside the macro. great for #284 which adds even more complicated async-stream(s).

good to go if CI passes.

Co-authored-by: Joonas Koivunen <joonas@equilibrium.co>
2020-08-06 12:27:54 +00:00
ljedrz
cb0fa7b3a4 chore: update the domain dep
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-06 14:08:36 +02:00
bors[bot]
9099945a57
Merge #289
289: Ignore a flaky Kademlia test r=koivunej a=ljedrz

`#[ignore]` that Kademlia test that likes to randomly fail due to being dependent on actual network.

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-06 11:52:52 +00:00