Commit Graph

646 Commits

Author SHA1 Message Date
Joonas Koivunen
4101383855 refactor: move try_only_named.. to support 2020-07-08 19:24:17 +03:00
Joonas Koivunen
087776bda1 refactor: try_only_named_multipart out of put_block 2020-07-08 19:17:58 +03:00
Joonas Koivunen
06ba40e8f2 fix: add descriptive error for LengthRequired 2020-07-08 16:03:10 +03:00
Joonas Koivunen
0a9ee274ca temp: upgrade to mpart-async with fix 2020-07-08 13:52:01 +03:00
Joonas Koivunen
76486f8ecb fix: rewrite block/put to support streaming multiparts
streaming multiparts are something we dont yet support, which is keeping
us back in an earlier version of js-ipfs-http-client for conformance
tests.
2020-07-08 13:52:01 +03:00
bors[bot]
f757458474
Merge #223
223: Arc-ify the Ipfs struct r=koivunej a=ljedrz

This allows us to pass the central `Ipfs` object around when needed, improving what we can do within its child objects. In addition:
- `impl Default for Bitswap` (now possible)
- replace `Arc<Repo>` with a reference to `Ipfs` in `IpldDag` and `Ipns`, which are now created on-the-fly when needed
- poll for `BitswapEvent`s using `NetworkBehaviourEventProcess`
- un`Arc` the `Repo`

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-07-06 10:01:53 +00:00
ljedrz
11cb434d68 refactor: create Dag and Ipns on the fly
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-06 11:32:45 +02:00
ljedrz
d5f6f5e300 chore: add a FIXME from the former PR
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-06 11:32:45 +02:00
ljedrz
872913ce3f refactor: remove Bitswap.io_events, use NetworkBehaviourEventProcess for BitswapEvents
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-06 11:32:45 +02:00
ljedrz
8b618aab7c refactor: wrap Ipfs in an Arc
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-06 11:32:45 +02:00
bors[bot]
65f884c849
Merge #221
221: remove Strategy, improve event handling r=koivunej a=ljedrz

The scope of changes is far greater than I've anticipated, but bear with me.

Key, planned changes:
- remove `bitswap::strategy`
- move `bitswap::Stats` to `ledger`
- introduce `BitswapEvents` emitted by `Bitswap::poll`
- move `Message` into `Ledger`

Drive-by changes forced by current design:
- wrap `Stats` in an `Arc`, make individual stats `AtomicU64`
- add a `Bitswap.queued_blocks` collection populated from `IpfsFuture::poll`
- make some child `bitswap` objects `pub` or `pub(crate)`
- update `Stats` from `IpfsFuture::poll` (good causality-wise, bad encapsulation-wise)
- introduce a `Bitswap.io_events` sender and its `IpfsFuture.bitswap_events` counterpart receiver

Observations:
- `Repo` is something we want to have access to both in `bitswap` (in order to find `want`ed `Block`s) and in `Ipfs` (to handle the higher-level API requests)
- `Ledger`s for the different `PeerId`s contain `Stats` that are updated at `IpfsFuture` level
- `BitswapEvents` don't currently result in anything of interest, due to `Behaviour` not containing the `Repo`

Key takeaway: we need to do some re-designing in order for key objects to have better access to one another.

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-07-06 09:10:43 +00:00
bors[bot]
933b52d97f
Merge #220
220: Initial file adder r=ljedrz a=koivunej

Creates multiblock balanced trees. Is quite slow and uses quite a lot of heap.

Co-authored-by: Joonas Koivunen <joonas@equilibrium.co>
Co-authored-by: Joonas Koivunen <joonas.koivunen@gmail.com>
2020-07-03 09:33:28 +00:00
ljedrz
33935bc109 refactor: better ledger stat handling
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-03 11:25:19 +02:00
Joonas Koivunen
2553de32de refactor: go back to moving builder 2020-07-03 12:20:58 +03:00
Joonas Koivunen
2d31efafb2 chore: changelog 2020-07-03 12:20:58 +03:00
Joonas Koivunen
77657806d7 doc: enhance, document lore, suggestions
Co-authored-by: ljedrz <ljedrz@users.noreply.github.com>
2020-07-03 12:20:58 +03:00
Joonas Koivunen
13e5036126 refactor: use Default::default() struct init 2020-07-03 12:20:58 +03:00
Joonas Koivunen
28065973ad refactor: name the 4-tuple as Link 2020-07-03 12:20:58 +03:00
Joonas Koivunen
a409095509 doc: suggestions from code review
Co-authored-by: ljedrz <ljedrz@users.noreply.github.com>
2020-07-03 12:20:58 +03:00
Joonas Koivunen
c47bfc1933 fix: debug intermediate levels as well 2020-07-03 12:20:58 +03:00
Joonas Koivunen
8912649d1d chore: clippy warning 2020-07-03 12:20:58 +03:00
Joonas Koivunen
0ff594c156 refactor: complicate/simplify link debugging 2020-07-03 12:20:58 +03:00
Joonas Koivunen
e24eb1c97e refactor: remove unused result return type 2020-07-03 12:20:58 +03:00
Joonas Koivunen
d3bd086d43 add: FileAdderBuilder and enhanced docs 2020-07-03 12:20:58 +03:00
Joonas Koivunen
eb3a0bb844 refactor: allow one less copy with prechunked input 2020-07-03 12:20:58 +03:00
Joonas Koivunen
1c4f59a3e7 fix: do less work to speed up test 2020-07-03 12:20:58 +03:00
Joonas Koivunen
c13bd36dcc refactor: collector out of the adder 2020-07-03 12:20:58 +03:00
Joonas Koivunen
7bddf33313 refactor: simplify link flushing 2020-07-03 12:20:58 +03:00
Joonas Koivunen
bdeefb0d85 doc: adjust comments 2020-07-03 12:20:58 +03:00
Joonas Koivunen
791b0334d9 perf: reuse links and blocksizes
no noticeable changes in benchmarks
2020-07-03 12:20:58 +03:00
Joonas Koivunen
15073666b1 fix: the 174*256*1024 + 1 case 2020-07-03 12:20:58 +03:00
Joonas Koivunen
ae1976b3f3 add: failing test case 2020-07-03 12:20:58 +03:00
Joonas Koivunen
fec28f6422 chore: rustfmt 2020-07-03 12:20:58 +03:00
Joonas Koivunen
bf9ec07d04 fix: produce block for empty files 2020-07-03 12:20:58 +03:00
Joonas Koivunen
8310df38af chore: remove mistakenly added ignore 2020-07-03 12:20:58 +03:00
Joonas Koivunen
89387c475e refactor: update comments, reorder uses 2020-07-03 12:20:58 +03:00
Joonas Koivunen
f6e78c3ff7 refactor: move file adder under file 2020-07-03 12:20:58 +03:00
Joonas Koivunen
153d6e52ec doc: file adder 2020-07-03 12:20:58 +03:00
Joonas Koivunen
8d0422ba35 fix: example cidv1 output, buffer to size hint 2020-07-03 12:20:58 +03:00
Joonas Koivunen
4444a2301d fix: add FileAdder impl Debug 2020-07-03 12:20:58 +03:00
Joonas Koivunen
b5868ee3e8 refactor: remove indentation level 2020-07-03 12:20:58 +03:00
Joonas Koivunen
c54118f324 test: more tests to catch the chunking issue 2020-07-03 12:20:53 +03:00
Joonas Koivunen
9ece3b943f test: add test case for three layers 2020-07-03 12:20:31 +03:00
Joonas Koivunen
587fd14461 add: examples/add.rs, still bad output 2020-07-03 12:20:31 +03:00
Joonas Koivunen
6498436ae8 chore: make sha2 and multihash !dev deps
they are needed in the adder to create links.
2020-07-03 12:20:31 +03:00
Joonas Koivunen
36ad14b127 feat: initial file adder
Should be fixable to create trees for files.
2020-07-03 12:20:31 +03:00
ljedrz
dfe28f742b feat: remove the strategy from Bitswap
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-02 15:16:31 +02:00
bors[bot]
d38e4cc1d0
Merge #218
218: remove the Strategy generic & replace libipld with cid in bitswap r=koivunej a=ljedrz

- we only support a single strategy and the fact it is a generic complicates things; make it non-generic, which paves way to improving event handling
- `cid` is a smaller dep than `libipld` and we only use its bits in `bitswap`

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-07-01 14:36:24 +00:00
bors[bot]
4bef83db7e
Merge #219
219: Adding new dependency to README r=koivunej a=aphelionz

fixes #217 

...maybe

Co-authored-by: Mark Robert Henderson <henderson.mark@gmail.com>
2020-07-01 14:04:37 +00:00
ljedrz
78237faf43 feat: improve event handling
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-01 16:04:26 +02:00