Commit Graph

1092 Commits

Author SHA1 Message Date
Joonas Koivunen
f0bb5b41fd feat: create empty directories without metadata
this is the go-ipfs 0.5 level support for add and directories. next up
will be parsing the headers as unixfsv1.5 metadata and using those with
the directories *and* files.
2020-08-07 15:06:55 +03:00
Joonas Koivunen
ccd6bbe248 feat: build directory trees on /add 2020-08-07 15:06:53 +03:00
Joonas Koivunen
c59a4cbe22 refactor: try make /add streaming 2020-08-07 15:05:05 +03:00
Joonas Koivunen
31e2a5ee5a fix: remove unneeded mut in render_directory 2020-08-07 15:05:05 +03:00
Joonas Koivunen
de9247574d feat: generate actual dag-pb 2020-08-07 15:05:05 +03:00
Joonas Koivunen
fa16bfd397 feat: unixfs tree builder 2020-08-07 15:05:05 +03:00
bors[bot]
926e2f6347
Merge #292
292: Use tokio::test in http instead of its async_std counterpart r=koivunej a=ljedrz

As `tokio` is the async executor in `http`.

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-07 08:40:22 +00:00
ljedrz
d2016f0193 fix: use tokio::test in http instead of its async_std counterpart
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-07 09:27:15 +02:00
bors[bot]
70bccd8873
Merge #280
280: Patch a Bitswap leak r=koivunej a=ljedrz

We currently don't properly clean up after a connection to a `Bitswap` peer is closed, which leads to leaking `Ledger`s; funnily enough, it seems that just uncommenting a pre-existing line solves the issue. In addition, add a relevant test.

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-06 15:08:27 +00:00
bors[bot]
0adaf255c6
Merge #291
291: fix: add timeouts to connect_two tests r=ljedrz a=koivunej

gha windows was stuck on this. it's not nice since we cannot see any
testing output before the run is cancelled.

Co-authored-by: Joonas Koivunen <joonas@equilibrium.co>
2020-08-06 14:24:05 +00:00
Joonas Koivunen
3e2554aa47 fix: add timeouts to connect_two tests
gha windows was stuck on this. it's not nice since we cannot see any
testing output before the run is cancelled.
2020-08-06 17:20:47 +03:00
ljedrz
d20131077a fix: consider the possibility that a peer is no longer available in send_block
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-06 16:02:05 +02:00
ljedrz
b5c0dcf907 docs: comment on not dropping peer stats on disconnects
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-06 16:02:05 +02:00
ljedrz
576ee6b92e fix: tweak a comment (drive-by)
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-06 16:02:05 +02:00
ljedrz
2dfabdf66a fix: patch a Bitswap leak and test it
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-06 16:02:04 +02:00
bors[bot]
0fb84c0b49
Merge #281
281: Some Kademlia debugging r=koivunej a=ljedrz

These changes were sparked by the following 2 observations:
- the logs sometimes indicate that a Kademlia query was executed twice
- `finish_subscription` didn't always result in futures being awoken

While the former remains a mystery (its occurrence is not correlated with subscriptions, meaning it's either some polling issue that eludes me or a bug in `libp2p`), investigating the `kad`<>`Subscription` route yielded a few improvements, namely:
- tweaked `kad` log levels
- proper `SubscriptionFuture` handling in `put_block`
- improved `SubscriptionRegistry` logs and a small fix
- simpler `SubscriptionFuture` type handling (it always returned a `Result`, so now it's a default)
- a `debug_assert` checking that we don't trigger zero-wake cases in `finish_subscription` during tests
- improved `swarm_api` test (that sometimes caused issues with the new `debug_assert`, but could be improved regardless)

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-06 13:39:18 +00:00
bors[bot]
b99288b7b1
Merge #290
290: Some maintenance r=koivunej a=ljedrz

Some maintenance work so as not to conflict too much with the pending PRs:
- depend on cid instead of libipld where only Cid is used
- update the `domain` dep (big `Cargo.lock` wins)

cc #75 

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-06 13:23:45 +00:00
bors[bot]
47d2674d57
Merge #288
288: fix: update async-stream to 0.3 r=ljedrz a=koivunej

this uses the rust 1.45 features to remove recursion inside the macro. great for #284 which adds even more complicated async-stream(s).

good to go if CI passes.

Co-authored-by: Joonas Koivunen <joonas@equilibrium.co>
2020-08-06 12:27:54 +00:00
ljedrz
cb0fa7b3a4 chore: update the domain dep
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-06 14:08:36 +02:00
bors[bot]
9099945a57
Merge #289
289: Ignore a flaky Kademlia test r=koivunej a=ljedrz

`#[ignore]` that Kademlia test that likes to randomly fail due to being dependent on actual network.

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-06 11:52:52 +00:00
ljedrz
f5f6790062 chore: depend on cid instead of libipld where only Cid is used
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-06 11:38:27 +02:00
ljedrz
7759ee0e32 fix: ignore a flaky kad test
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-06 10:48:06 +02:00
Joonas Koivunen
5b6b5ddb1b fix: update async-stream to 0.3
this uses the rust 1.45 features to remove recursion inside the macro.
great for #284 which adds even more complicated async-stream(s).
2020-08-06 11:10:07 +03:00
ljedrz
e2f87bd32b fix: disable provide_block for now
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-05 14:08:16 +02:00
ljedrz
7e799f0e90 docs: comment on not logging the key in some kad queries
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-05 13:58:45 +02:00
ljedrz
77d9a6cc0e fix: comment on prove_block and only provide new blocks
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-05 11:58:16 +02:00
ljedrz
c3bcadf5d3 chore: remove an outdated comment, fix Debug for SubscriptionFuture
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-05 11:46:36 +02:00
bors[bot]
1720e16c94
Merge #283
283: Remove the rocksdb feature r=koivunej a=ljedrz

We no longer intend to pursue that feature; remove the related code.

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-05 08:51:18 +00:00
ljedrz
0d43557bab feat: remove the rocksdb feature
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-05 09:43:08 +02:00
ljedrz
61da270487 fix: improve the swarm_api test
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-04 16:49:31 +02:00
ljedrz
ffa0abe843 fix: tighten the SubscriptionRegistry, tweak related types
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-04 16:49:31 +02:00
ljedrz
9e90fa13c0 fix: tweak logs and cancel conditions for Subscriptions
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-04 14:53:49 +02:00
ljedrz
58c904072f fix: wait for Kademlia in put_block
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-04 14:06:28 +02:00
ljedrz
ed20a189e7 chore: tweak some kademlia log levels
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-04 13:20:18 +02:00
bors[bot]
6b6ddcdce2
Merge #278
278: Update filetime r=ljedrz a=c410-f3r

Sets `0.2.12` as the minimum required version because of https://github.com/alexcrichton/filetime/issues/62

Co-authored-by: Caio <c410.f3r@gmail.com>
2020-08-03 18:07:31 +00:00
Caio
f69f2d25b8 Update filetime 2020-08-03 14:47:17 -03:00
bors[bot]
e8d600b5b6
Merge #277
277: Add the means to connect by PeerId r=ljedrz a=ljedrz

I've encountered this while working on improved bitswap <> kademlia interop - we still can't easily connect using `PeerId`. Improve this by introducing an `enum ConnectionTarget` that is either a `Multiaddr` or a `PeerId` and adjusting the `SwarmApi.connect_registry` accordingly. There's also a new related test.

As a drive-by, adjust some `tracing` features that are among the defaults but we'd still enjoy having 😃.

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-03 15:13:53 +00:00
ljedrz
38b39747bb feat: introduce the means to connect by PeerId
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-03 14:24:01 +02:00
ljedrz
e929cdf28b fix: extend tracing features
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-03 13:23:45 +02:00
bors[bot]
d4c4f92e4e
Merge #276
276: Remove unused dependencies r=koivunej a=c410-f3r

First commit only adds `default-features = false` where applicable while the second commit future-proofs possible upstream modifications for every single crate.

```
* bitswap

cargo build: 199 -> 179
cargo check: 212 -> 191
cargo test: 201 -> 181


* ipfs

cargo build: 419 -> 341
cargo check: 433 -> 354
cargo test: 445 -> 359


* ipfs-http

cargo build: 518 -> 408
cargo check: 556 -> 429
cargo test: 522 -> 412
```

By looking into the differences, it is possible to see astonishing results across all operations.

Co-authored-by: Caio <c410.f3r@gmail.com>
2020-08-03 08:54:33 +00:00
Caio
15ff8885b2 Fix android build and include unixfs crate 2020-08-01 14:31:11 -03:00
Caio
94d9b0a358 Future-proof unused dependencies 2020-08-01 13:44:27 -03:00
Caio
d3bd3f649f Use default-features = false where applicable 2020-08-01 13:29:48 -03:00
bors[bot]
e224b715ab
Merge #274
274: fix bogus empty bitswap messages r=ljedrz a=ljedrz

We've been seeing `(empty message)` bitswap for a while, but these logs were bogus - this PR fixes those, and adds a small bitswap stress test that can be used for future bitswap debugging.

Also removes some noisy logging and, as a drive-by, fixes a subscription-related log.

Co-authored-by: ljedrz <ljedrz@gmail.com>
Co-authored-by: ljedrz <ljedrz@users.noreply.github.com>
2020-07-31 14:43:32 +00:00
ljedrz
dc48504d1c chore: add a comment to the bitswap stress test
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-31 16:42:50 +02:00
ljedrz
0e45c71c7f
chore: expand a bitswap comment
Co-authored-by: Joonas Koivunen <joonas.koivunen@gmail.com>
2020-07-31 16:35:42 +02:00
ljedrz
c0524d5791 fix: don't report bogus empty bitswap messages
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-31 16:12:55 +02:00
ljedrz
0d74cea3df feat: add a bitswap stress test
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-31 15:52:56 +02:00
bors[bot]
9b316a3e27
Merge #268
268: ci: caching updates r=aphelionz a=koivunej

node_modules: It's quite slow on windows (about 3min), linux (50s) and mac is in the between. This is a bit hacky given the patching and whatnot, but I think the hashFiles combination is good.

~/.cargo: this was never cached, but now it should be cached correctly `hashFiles(Cargo.lock)`.

Co-authored-by: Joonas Koivunen <joonas@equilibrium.co>
2020-07-31 12:55:10 +00:00
ljedrz
dc4cea6df5 refactor: less noisy upgrade_outbound
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-31 14:45:49 +02:00