Commit Graph

72 Commits

Author SHA1 Message Date
Joonas Koivunen
4421608284 doc: remove out of date fetch_and_cat comment 2020-10-08 10:16:41 +03:00
Joonas Koivunen
d026fff0fe
doc: suggestions from code review
Co-authored-by: ljedrz <ljedrz@users.noreply.github.com>
2020-10-07 16:55:08 +03:00
Joonas Koivunen
2b496ee989 refactor: showcase restore_bootstrappers in fetch_and_cat 2020-10-07 16:03:22 +03:00
Joonas Koivunen
f6ec5fa8f1 fix: examples and tests 2020-09-23 14:15:34 +03:00
ljedrz
c1fcc6be34 chore: move the example from README to examples
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-09-22 16:19:16 +02:00
Joonas Koivunen
ae646201c7
doc: suggestions from code review
Co-authored-by: ljedrz <ljedrz@users.noreply.github.com>
2020-09-22 16:28:17 +03:00
Joonas Koivunen
23cc1bf288 remove ipfs_bitswap_test
the example has long been superceded by both fetch_and_cat and
integration tests.
2020-09-22 16:00:31 +03:00
Joonas Koivunen
854d3afad7 doc: commenting pass on fetch_and_cat 2020-09-22 16:00:31 +03:00
Joonas Koivunen
65c3f48cba remove obsolete examples
client1 and client2 were a pair of two peers working together to form up
an example where one client creates an ipld document and the other one
downloads it, reading IPLD projections. the bitswap side is already done
elsewhere and the nature of use having to run two clients, there just
wasn't an easy way around without running the two nodes in the same
process. if that was done, then the example would be the same as
tests.
2020-09-22 16:00:31 +03:00
ljedrz
d1773c4a0f chore: remove the ipns example
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-09-22 10:29:34 +02:00
ljedrz
a29f3d328c refactor: reuse an existing resolve function
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-09-02 10:07:14 +02:00
Joonas Koivunen
f2ddb18b6c
doc: suggestions from code review
Co-authored-by: ljedrz <ljedrz@users.noreply.github.com>
2020-08-26 18:12:05 +03:00
Joonas Koivunen
b7a1c70324 example(fetch_and_cat): disable mdns 2020-08-26 18:06:05 +03:00
Joonas Koivunen
4af69120d4 example(fetch_and_cat): add argument: peer to connect to 2020-08-26 18:06:05 +03:00
Joonas Koivunen
e2efa5236a doc: suggestions from code review
Co-authored-by: ljedrz <ljedrz@users.noreply.github.com>
2020-08-25 14:07:29 +03:00
Joonas Koivunen
5f1c958867 examples: add more progress output to fetch_and_cat 2020-08-25 12:36:37 +03:00
Joonas Koivunen
e04dd9a6f0 feat: propagate path resolving to ipfs::unixfs::cat level
solved this wrong by isolating it first at http level, however now at
the ipfs::unixfs::cat level the http implementation is looking small as
it should be.

touched some error types and converted them to thiserror.
2020-08-25 12:36:37 +03:00
ljedrz
398aa58877 chore: import reordering with cargo fmt
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-13 16:15:45 +02:00
ljedrz
8be75f7a4a feat: change the async executor to tokio
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-13 16:15:14 +02:00
ljedrz
b1bc672559 chore: merge and adapt the libipld dependency
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-10 13:15:11 +02:00
ljedrz
f8674a0504 refactor: remove the generic from IpfsOptions, RepoOptions and SwarmOptions
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-10 11:49:43 +02:00
ljedrz
f5f6790062 chore: depend on cid instead of libipld where only Cid is used
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-08-06 11:38:27 +02:00
ljedrz
e820e7a5e8 feat: keep a node's Span instead of its name
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-30 17:32:25 +02:00
ljedrz
a284c1f841 feat: replace log with tracing
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-29 14:50:00 +02:00
Joonas Koivunen
394d003efa chore: cargo fmt 2020-06-26 16:17:03 +03:00
Joonas Koivunen
27016b9901 refactor: remove unused use 2020-06-26 12:45:51 +03:00
Joonas Koivunen
1ea73c0c72 chore: add guidance to example, remove file creation 2020-06-26 12:41:24 +03:00
ljedrz
a3b9b44e77 fix: make examples simple again 2020-06-25 10:11:27 +02:00
Joonas Koivunen
f935c7bc72 refactor: use unwrap_or_else 2020-06-11 19:10:46 +03:00
Joonas Koivunen
392840c0ad chore: cargo fmt 2020-06-10 20:32:58 +03:00
Joonas Koivunen
ffa747db68 refactor: resolve first block before streaming bodies
this needs to be done to return error when using cat on directories
2020-06-10 20:32:58 +03:00
Joonas Koivunen
36f3de7eb9 chore: cargo fmt 2020-06-08 15:03:34 +03:00
Joonas Koivunen
d43cf70eef refactor: manual test case into an example 2020-06-08 15:03:34 +03:00
Joonas Koivunen
cad93bc4ed refactor: add perhaps extra recursive_limit
could be that this started erroring only as I started working on the
fetch_and_cat example.
2020-06-08 15:03:34 +03:00
Joonas Koivunen
79985e4140
Add refs endpoint (#147)
A lot of code but going back and forth as well.

* fix: request logger name for good

* wip: initial broilerplate

* feat: ipfspath parsing and walking

* refactor: rewrite to walk the path before streaming

* fix: handling 0 for ipld link

* wip: get first tests to pass

still need to figure out the Send + Sync + 'static bound, issue is with
Sync, which might come from the refs used, get_block(&cid)..

this required some unsafety to be able to present the Stream as Send +
Sync + 'static as required by hyper. not 100% sure if that pin_project
is ok to do, but the only interfaces exposed are callable as futures so
it shouldn't be too bad.

* feat: format string support
* add: manual RefsOptions deserialization to have multiple `arg`

* fix: resolve multiple paths

I had not originally noticed that we need to do the walk from multiple
paths.

* add: dag-pb specific ipfspath walking

* add: make IpfsPath better debuggable

* wip: adjust logging

* add: yield ipld parsing failed err's

* add: "specialize" ipld_links for dag-pb

* feat: support visiting unique nodes

* fix: instead of stopping, filter out too deep elements

also, this should be done when expanding the nodes, not at this stage.

* refactor: reorg and document

originally it was easier to write everything to the same file, but it
gets difficult to read after 1000 lines. while reorganising stuff into
more tight modules, some direct calls were changed not to be needed,
like RefsOptions creating the EdgeFormatter; the EdgeFormatter is not
created in the refs function itself.

chopped off parts:

 * RefsOptions stuff to refs/options.rs
 * EdgeFormatter and the string format to refs/format.rs
 * Unshared to refs/unshared.rs
 * HandledErr, StreamResponse to refs/support.rs

* refactor: minor cleanup and docs

* fix: refs tests, enable unique test

* fix: use specific rust-ipld version with cbor fix

the situation is not good, as it would make sense to re-export libipld
from ipfs as parts are already re-exported but with strange names.

* refactor: allow dead_code StringError::new

* refactor: use the Edge for refs-local as well

* chore: cargo fmt and reorder tests

* doc: replace fixme with confirmed

* add: port tests for ipfspath

* fix: unignore good_but_unsupported test

* add: test cases for slash dedup per discussion

* add: walk dag-cbor tests for IpfsPath

* add: dag-pb path walking test

* add: tests for walking over a cbor link

* add: test case for dapb link listing

* fix: refs_local test, move all tests under mod test

* fix: remove unwrap from refs-local

kept the return value as a stream. it'll at least save us the trouble of
concatenating all of the values into a big vector.
2020-04-16 16:39:48 +03:00
Joonas Koivunen
97b5817381 refactor: remove unnecessary mut refs
unsure why this isn't picked up by clippy.
2020-04-15 09:54:53 -04:00
Mark Henderson
d3912d511b fix: examples and unused" 2020-04-14 08:04:06 -04:00
David Craven
8889b0de12 Fix ci. 2020-03-21 13:25:32 -03:00
David Craven
180baff77f Move examples to separate crate. 2020-03-21 12:10:32 -03:00
Joonas Koivunen
cb664e11f8 fix: go back to using futures channels
we cannot simply replace async aware channels with crossbeam channels,
which cannot wake us up. keeping the channel buffer size at 1 to catch
any errors.
2020-03-17 10:34:43 +02:00
David Craven
1428034150 Move bitswap into it's own crate. 2020-03-13 13:53:55 +01:00
David Craven
ae7ed3c852 Use rust-ipld. 2020-03-10 09:15:45 +01:00
David Craven
8419fe8438 Test README on ci. 2020-03-06 17:38:37 +01:00
David Craven
34fab7c774 Run rustfmt. 2020-03-03 13:00:21 +01:00
David Craven
edd22040d0 Perform some cleanup and update deps. 2020-03-03 13:00:21 +01:00
Rüdiger Klaehn
e337b85285 Fix compile errors 2020-01-10 16:28:31 +01:00
Joonas Koivunen
cf381aa559 refactor rest of clippy warnings 2020-01-08 15:21:41 +01:00
Joonas Koivunen
77c9646640 refactor restore tests
finally converged to a `crate::tests::async_test` wrapper for invoking
`tokio::run` **and** getting failures out of the test. later tokios
should include test wrappers.
2019-12-05 20:14:24 +01:00
Joonas Koivunen
12ee9d5512 refactor cargo update and some of the fixes
- add async_trait dep for repo
 - use async fn everywhere possible
 - update toolchain to semi-latest
 - update futures 0.3
 - update libp2p 0.13
 - get rid of rocksdb for now
 - allow warnings and disable bare_trait_objects for protoc generated
 - wasteful vec workaround (shouldve used MultihashRef?)
 - stop using drain_filter (possibly correctly?)

Note: this changes the generated code
Note: feature(try_trait) remains
2019-12-05 20:14:24 +01:00
David Craven
f436bb4cf8
Fix parse deprecation warning by using parse_filters. 2019-03-08 15:53:43 +01:00