Commit Graph

22 Commits

Author SHA1 Message Date
Andrei Aaron
ce4924f841
refactor: rename go module from zotregistry.io/zot to zotregistry.dev/zot (#2187)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2024-01-31 20:34:07 -08:00
LaurentiuNiculae
3f97f878fd
feat(cve): add option to exclude string from cve search (#2163)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2024-01-19 12:59:42 -08:00
Andrei Aaron
18aa975ae2
feat(CVE): add CVE severity counters to returned images and CVE list calls (#2131)
For CLI output is similar to:

CRITICAL 0, HIGH 1, MEDIUM 1, LOW 0, UNKNOWN 0, TOTAL 2

ID                SEVERITY  TITLE
CVE-2023-0464     HIGH      openssl: Denial of service by excessive resou...
CVE-2023-0465     MEDIUM    openssl: Invalid certificate policies in leaf...

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-12-13 09:16:31 -08:00
peusebiu
7642e5af98
fix(scheduler): fix data race (#2085)
* fix(scheduler): data race when pushing new tasks

the problem here is that scheduler can be closed in two ways:
- canceling the context given as argument to scheduler.RunScheduler()
- running scheduler.Shutdown()

because of this shutdown can trigger a data race between calling scheduler.inShutdown()
and actually pushing tasks into the pool workers

solved that by keeping a quit channel and listening on both quit channel and ctx.Done()
and closing the worker chan and scheduler afterwards.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>

* refactor(scheduler): refactor into a single shutdown

before this we could stop scheduler either by closing the context
provided to RunScheduler(ctx) or by running Shutdown().

simplify things by getting rid of the external context in RunScheduler().
keep an internal context in the scheduler itself and pass it down to all tasks.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>

---------

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-12-11 10:00:34 -08:00
LaurentiuNiculae
79e14027ee
refactor(test): add lint rule for messages starting with the component (#2045)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-12-08 10:05:02 +02:00
peusebiu
6222dae1f0
feat(scheduler): gracefully shutdown (#1951)
wait for workers to finish before exiting

should fix tests reporting they couldn't remove rootDir because it's being
written by tasks

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-11-24 00:40:10 -08:00
LaurentiuNiculae
92837c2bcb
refactor(log): replace panics with log fatal or log panic functions (#1723)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-24 00:38:36 -08:00
LaurentiuNiculae
83f287d1f6
feat(cli): add command to interogate the server version and other details (#1709)
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
2023-11-22 19:25:21 +02:00
Andreea Lupu
d5065513f5
feat: add support for oci1.1 cosign signatures(using referrers) (#1963)
- Cosign supports 2 types of signature formats:

	1. Using tag -> each new signature of the same manifest is
	added as a new layer of the signature manifest having that
	specific tag("{alghoritm}-{digest_of_signed_manifest}.sig")

	2. Using referrers -> each new signature of the same manifest is
	added as a new manifest

- For adding these cosign signature to metadb, we reserved index 0 of the
list of cosign signatures for tag-based signatures. When a new tag-based
signature is added for the same manifest, the element on first position
in its list of cosign signatures(in metadb) will be updated/overwritten.
When a new cosign signature(using referrers) will be added for the same
manifest this new signature will be appended to the list of cosign
signatures.

Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-11-06 14:09:39 -08:00
LaurentiuNiculae
56ad9e6707
refactor(metadb): improve UX by speeding up metadb serialize/deserialize (#1842)
Use protocol buffers and update the metadb interface to better suit our search needs

Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Signed-off-by: Laurentiu Niculae <niculae.laurentiu1@gmail.com>
Co-authored-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-30 13:06:04 -07:00
Alexei Dodon
d2fbd273ba
fix: tests refactoring (#1950)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-10-26 11:20:39 +03:00
Ramkumar Chinchani
1675f30d4a
ci: update golangci-lint version (#1834)
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
2023-10-20 17:27:04 +03:00
Andreea Lupu
fc2380b57b
fix: add support for uploaded index when signing using notation (#1882)
ci(notation): update to latest notation version
fix(sync): add layers info when syncing signatures

Signed-off-by: Andreea-Lupu <andreealupu1470@yahoo.com>
2023-10-12 18:45:20 -07:00
Alexei Dodon
044ea85279
fix: running tests locally fails (#1879)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-10-05 14:34:50 +03:00
Andrei Aaron
ca1c3288cf
refactor(test): make sure cli tests are not internal unless they need to be (#1878)
As part of this change searchConfig needed to be exported,
as it was passed as a parameter to exported functions

At this moment most of the tests remaining internal depend on the mock service.
The interface it implements has unexported methods.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-10-03 11:15:39 -07:00
Andrei Aaron
99e29c0f46
refactor(tests): Migrate some of the older tests to the new image-utils library (#1863)
Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-10-02 11:10:43 -07:00
Andrei Aaron
ba6f347d8d
refactor(pkg/test): split logic in pkg/test/common.go into multiple packages (#1861)
Which could be imported independently. See more details:
1. "zotregistry.io/zot/pkg/test/common" - currently used as
   tcommon "zotregistry.io/zot/pkg/test/common" - inside pkg/test
   test "zotregistry.io/zot/pkg/test/common" - in tests
   . "zotregistry.io/zot/pkg/test/common" - in tests
Decouple zb from code in test/pkg in order to keep the size small.

2. "zotregistry.io/zot/pkg/test/image-utils" - curently used as
   . "zotregistry.io/zot/pkg/test/image-utils"

3. "zotregistry.io/zot/pkg/test/deprecated" -  curently used as
   "zotregistry.io/zot/pkg/test/deprecated"
This one will bre replaced gradually by image-utils in the future.

4. "zotregistry.io/zot/pkg/test/signature" - (cosign + notation) use as
   "zotregistry.io/zot/pkg/test/signature"

5. "zotregistry.io/zot/pkg/test/auth" - (bearer + oidc)  curently used as
   authutils "zotregistry.io/zot/pkg/test/auth"

 6. "zotregistry.io/zot/pkg/test/oci-utils" -  curently used as
   ociutils "zotregistry.io/zot/pkg/test/oci-utils"

Some unused functions were removed, some were replaced, and in
a few cases specific funtions were moved to the files they were used in.

Added an interface for the StoreController, this reduces the number of imports
of the entire image store, decreasing binary size for tests.
If the zb code was still coupled with pkg/test, this would have reflected in zb size.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-09-27 11:34:48 -07:00
peusebiu
1df743f173
fix(gc): sync repodb when gc'ing manifests (#1819)
fix(gc): fix cleaning deduped blobs because they have the modTime of
the original blobs, fixed by updating the modTime when hard linking
the blobs.
fix(gc): failing to parse rootDir at zot startup when using s3 storage
because there are no files under rootDir and we can not create empty dirs
on s3, fixed by creating an empty file under rootDir.

Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-09-22 11:51:20 -07:00
Andrei Aaron
7c78f80a96
feat(cve): implement CVE scanning as background tasks (#1833)
1. Move existing CVE DB download generator/task login under the cve package
2. Add a new CVE scanner task generator and task type to run in the background, as well as tests for it
3. Move the CVE cache in its own package
4. Add a CVE scanner methods to check if an entry is present in the cache, and to retreive the results
5. Modify the FilterTags MetaDB method to not exit on first error
This is needed in order to pass all tags to the generator,
instead of the generator stopping at the first set of invalid data
6. Integrate the new scanning task generator with the existing zot code.
7. Fix an issue where the CVE scan results for multiarch images was not cached
8. Rewrite some of the older CVE tests to use the new image-utils test package
9. Use the CVE scanner as attribute of the controller instead of CveInfo.
Remove functionality of CVE DB update from CveInfo, it is now responsible,
as the name states, only for providing CVE information.
10. The logic to get maximum severity and cve count for image sumaries now uses only the scanner cache.
11. Removed the GetCVESummaryForImage method from CveInfo as it was only used in tests

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-09-22 11:49:17 -07:00
Alexei Dodon
4e04be420e
refactor(cli): Move cmdflags package under pkg/cli/client (#1840)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-09-22 16:33:18 +03:00
Andrei Aaron
bcdd9988f5
fix(cve): cummulative fixes and improvements for CVE scanning logic (#1810)
1. Only scan CVEs for images returned by graphql calls
Since pagination was refactored to account for image indexes, we had started
to run the CVE scanner before pagination was applied, resulting in
decreased ZOT performance if CVE information was requested

2. Increase in medory-cache of cve results to 1m, from 10k digests.

3. Update CVE model to use CVSS severity values in our code.
Previously we relied upon the strings returned by trivy directly,
and the sorting they implemented.
Since CVE severities are standardized, we don't need to pass around
an adapter object just for pagination and sorting purposes anymore.
This also improves our testing since we don't mock the sorting functions anymore.

4. Fix a flaky CLI test not waiting for the zot service to start.

5. Add the search build label on search/cve tests which were missing it.

6. The boltdb update method was used in a few places where view was supposed to be called.

7. Add logs for start and finish of parsing MetaDB.

8. Avoid unmarshalling twice to obtain annotations for multiarch images.

Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
2023-09-17 15:12:20 -07:00
Alexei Dodon
f58597ade9
refactor: Reduce zli binary size (#1805)
Signed-off-by: Alexei Dodon <adodon@cisco.com>
2023-09-15 15:17:01 -07:00