Commit Graph

984 Commits

Author SHA1 Message Date
ljedrz
42da57f419 fix: change async_std to tokio
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-13 17:50:06 +02:00
ljedrz
b08c78d587 test: tweak the subscription stress test
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-13 17:31:37 +02:00
ljedrz
c8b885acd2 docs: document the subscription stress test
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-13 17:31:37 +02:00
ljedrz
10bfeac0bf feat: add a subscription stress test
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-13 17:31:37 +02:00
bors[bot]
143c3dad93
Merge #307
307: Use tokio instead of async_std r=koivunej a=ljedrz

We already use it in `ipfs-http`, our dependencies either already use it or are compatible with it and it has some nice features that we'd like to use.

Note: the `async` tests now specify `max_threads = 1` in order to make sure the multi-threaded executor is not being lenient about any hidden bugs.

fixes https://github.com/rs-ipfs/rust-ipfs/issues/275

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-13 15:04:45 +00:00
ljedrz
7ff0a3ccfb fix: ignore resolve tests due to Windows errors
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-13 16:47:10 +02:00
ljedrz
c930f651fd test: make async tests use only 1 thread
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-13 16:16:20 +02:00
ljedrz
4001e01a44 refactor: remove unused tokio features
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-13 16:15:47 +02:00
ljedrz
9e26bffbba chore: cargo clippy
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-13 16:15:47 +02:00
ljedrz
398aa58877 chore: import reordering with cargo fmt
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-13 16:15:45 +02:00
ljedrz
8be75f7a4a feat: change the async executor to tokio
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-13 16:15:14 +02:00
bors[bot]
4c90dd15a9
Merge #306
306: Cleanup old unixfs api r=koivunej a=koivunej

This removes the previous API of:

* ipfs::Ipfs::add
* ipfs::ipfs::get
* ipfs::unixfs::File

Rationale for the removal is that ipfs-unixfs (under `unixfs/`, re-exported as `ipfs::unixfs::ll`) provides ability to do the similar use cases more or less. It does not however, provide means to have a nice api and persist stuff in an async way which is something we are lacking at the moment. I will probably file an issue for this API deficiency once I am through this.

Co-authored-by: Joonas Koivunen <joonas@equilibrium.co>
2020-08-13 13:54:51 +00:00
Joonas Koivunen
7dd466953e refactor: clippy 2020-08-13 16:48:54 +03:00
Joonas Koivunen
ac149f4813 fix: update test case
there was a discrepency with the old and new way which took some time to
realize. if this destroys anyones workflow I hope we can find a way to
accomondiate you :)
2020-08-13 16:26:32 +03:00
Joonas Koivunen
b5e79da0a6 fix: remove old single block unixfs api 2020-08-13 14:24:56 +03:00
bors[bot]
fbf4d55100
Merge #300
300: use noise authentication with the XX handshake r=koivunej a=ljedrz

The new `0.23` release of `libp2p` seems to have fixed the `noise` incompatibility issues with `go-ipfs`, so update it, drop `SECIO` and use `noise-xx` instead.

While I've successfully tested this both ways against a `go-ipfs-0.6` node, an extra manual test won't hurt 😉.

fixes https://github.com/rs-ipfs/rust-ipfs/issues/75
fixes https://github.com/rs-ipfs/rust-ipfs/issues/175

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-12 13:46:16 +00:00
bors[bot]
b17aae2c06
Merge #304
304: Work around the rare zero wakers CI failure r=koivunej a=ljedrz

As far as I can tell this only happens to Kademlia queries and thus might be a `libp2p` bug; disable the sanity debug check for them to verify it.

Also, remove an outdated comment as a drive-by.

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-12 13:26:09 +00:00
ljedrz
8eab625653 fix: exclude KadQueries from the debug_assert in subscription
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-12 13:58:38 +02:00
ljedrz
77d80e26cb chore: remove an outdated comment
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-12 13:57:47 +02:00
ljedrz
ce45849117 chore: add more debug info to the subscription debug_assert
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-12 13:57:47 +02:00
bors[bot]
bcccc532d0
Merge #303
303: Allow connecting using multiaddr/p2p/peerid r=koivunej a=ljedrz

Also add related tests and tweak `connect_two` tests a bit.

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-12 10:34:18 +00:00
ljedrz
50dd210719 test: tweak two node connection tests
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-12 12:04:15 +02:00
ljedrz
cc12650d17 feat: allow connecting using /Multiaddr/p2p/PeerId addresses
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-12 12:04:15 +02:00
bors[bot]
efa44fa568
Merge #297
297: impl<T: Clone, E: Clone> Clone for Subscription<T, E> r=koivunej a=ljedrz

My suspects are either the enabled `tracing_subscriber` or the lock over `SubscriptionRegistry` taken in `get_subscriptions`; I've adjusted those and tightened the test's deadlines and wasn't able to trigger the issue while stress-testing. It might not suffice yet, but those changes won't hurt.

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-12 09:45:20 +00:00
bors[bot]
47322befb7
Merge #296
296: test duplicate connection attempts r=ljedrz a=ljedrz

While attempting to connect to a pre-connected `Multiaddr` didn't cause issues, duplicate `PeerId` attempts caused a hang.

I started by ensuring that the `PeerId` connection subscriptions are handled in the same manner as the `Multiaddr` ones. This, however, was insufficient. I then moved the creation of connection subscriptions before the connection event was pushed onto the swarm event stack, but it was not enough too. The final measure was to just ignore duplicate connection target attempts, which now doesn't even create a `Subscription`, but just causes the top-level call to return an error.

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-12 09:14:59 +00:00
ljedrz
a0b777d542 fix: ignore connect_two_nodes_by_peer_id, add relevant comments
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-12 11:13:00 +02:00
ljedrz
58a966a24a feat: test duplicate connection attempts
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-12 10:04:04 +02:00
bors[bot]
5755e8ba2a
Merge #301
301: Put "Rust-IPFS contributors" as package authors r=koivunej a=ljedrz

The usual convention for Rust crates that are a collaborative, open source effort is to put `<project> contributors` under the `[package] authors` in the `toml`.

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-11 15:02:37 +00:00
ljedrz
a9712ef740 chore: put Rust-IPFS contributors as authors
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-11 15:14:43 +02:00
ljedrz
cba4feb7e1 feat: impl<T: Clone, E: Clone> Clone for Subscription<T, E>; shorten the lock in a test method
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-11 15:10:34 +02:00
ljedrz
83447e8ec3 feat: use noise authentication with the XX handshake
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-11 13:44:27 +02:00
bors[bot]
4d4f3ca6b9
Merge #299
299: fix: go back to std::sync::Mutex r=koivunej a=koivunej

Turns out #174 was wrong, as the alternative required sprinkling block_on which made the whole thing worse. We shouldn't risk a deadlock with the mutex as we are not (can not) holding it across awaits.

Unwrap as we are not expecting subscriptions to panic and poison the mutex.

Co-authored-by: Joonas Koivunen <joonas@equilibrium.co>
2020-08-11 11:17:08 +00:00
Joonas Koivunen
c95a9bb862 fix: first drop mutex guard, then assert
the assert while holding the guard poisons the mutex and that causes
drop to panic because of the poisoned mutex, by design; see last
commit. by shortening the scope of the mutex guard we should once again
see more clear build failures :)
2020-08-11 13:51:17 +03:00
Joonas Koivunen
9e695e02cb fix: go back to std::sync::Mutex
Turns out #174 was wrong, as the alternative required sprinkling
block_on which made the whole thing worse. we shouldn't risk a deadlock
with the mutex as we are not (can not) holding it across awaits.

unwrap as we are not expecting subscriptions to panic and poison the
mutex.
2020-08-11 13:03:45 +03:00
bors[bot]
1b1aab339b
Merge #295
295: Merge and adapt the libipld dependency r=koivunej a=ljedrz

Our [libipld](https://crates.io/crates/libipld) dependency has been out of date with _its_ dependencies for a while now and since we don't need to maintain a shadow fork of the entire repo in order to be able to keep up, we might as well merge the bits we need until the upstream repo(s) are updated and we can depend on them directly again.

All the copied (with minor adjustments) code resides in the `ipld` module where the source repo is credited.

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-10 12:31:35 +00:00
ljedrz
b1bc672559 chore: merge and adapt the libipld dependency
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-10 13:15:11 +02:00
bors[bot]
16ae43011b
Merge #286
286: Ipfs object tweaks r=ljedrz a=ljedrz

Removes `SwarmTypes`, drops the generic from `IpfsOptions`, `RepoOptions` and `SwarmOptions` and simplifies the creation of `Ipfs`/`UnitializedIpfs` via more pattern-matching `move`s.

Rationale: since we already need to specify the type in order to create `IpfsOptions`, we might as well do that directly for `Ipfs` instead, without dragging the generic through all the options that don't even use it (they just carry `PhantomData`).

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-08-10 10:24:42 +00:00
ljedrz
f8674a0504 refactor: remove the generic from IpfsOptions, RepoOptions and SwarmOptions
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-10 11:49:43 +02:00
ljedrz
b95cb29650 refactor: simplify Ipfs object creation
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-10 11:46:18 +02:00
ljedrz
8e42d203c0 refactor: remove SwarmTypes
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-10 11:46:18 +02:00
bors[bot]
e00198a4d6
Merge #284
284: unixfs: feat tree building r=koivunej a=koivunej

Includes a `BufferingTreeBuilder` which kind of follows the js-ipfs implementation: a *complete* tree is built from path segments and later it's converted into a dag-pb by an "iterator" which walks the tree in post order, converting (rendering) the most nested elements first. Communication back to the "parent" is done using a `HashMap<u64, Vec<Option<(String, Cid, usize)>>>` which relies on initial ordering provided by `BufferingTreeBuilder` by using BTreeMap's on each level.

Co-authored-by: Joonas Koivunen <joonas@equilibrium.co>
Co-authored-by: Joonas Koivunen <joonas.koivunen@gmail.com>
2020-08-10 09:18:51 +00:00
Joonas Koivunen
aaabc37562 doc: suggestions and notes from code review
Co-authored-by: ljedrz <ljedrz@users.noreply.github.com>
2020-08-10 12:04:03 +03:00
Joonas Koivunen
276e4b7441 bench: filter symlinks earlier 2020-08-10 11:45:58 +03:00
Joonas Koivunen
14c27fd1b8 bench: move buffer and fix path growing outside 2020-08-10 11:44:10 +03:00
Joonas Koivunen
665c0c4f85 doc: comment touch ups 2020-08-10 09:23:11 +03:00
Joonas Koivunen
bfc1bf167c refactor: rename put_file => put_link 2020-08-10 09:22:58 +03:00
Joonas Koivunen
9df95ad1e8 refactor: remove duplicated ingest-tar example, it remains as bench 2020-08-10 09:13:44 +03:00
Joonas Koivunen
7446810982 doc: minor docs and fmt 2020-08-09 22:24:36 +03:00
Joonas Koivunen
71b02cf370 chore: cargo keeps changing the index file, clippy
not sure what is the index file hash change.
2020-08-09 22:24:36 +03:00
Joonas Koivunen
5484f3ed8f fix: make the benchmark more generic 2020-08-09 22:24:36 +03:00