Make uploaded licence and incident files retrievable and persistent (BE 028) - #495
Open
Houda135 wants to merge 2 commits into
Open
Make uploaded licence and incident files retrievable and persistent (BE 028)#495Houda135 wants to merge 2 commits into
Houda135 wants to merge 2 commits into
Conversation
…BE 028) Uploaded files were written to disk but could not be fetched back, and were lost whenever the backend container was rebuilt. One upload path also had no size limit at all. - Add a named docker volume for app-backend/src/uploads so files survive container rebuilds. - Add GET /api/v1/documents/:id/file to retrieve a document's file. A guard may fetch their own, an admin may fetch anyone's. Employer licence access is out of scope, see the PR notes. - Resolve stored file references through utils/uploadPath.js, which keeps the final path inside the uploads directory so a stored "../" value cannot reach other files on the server. - Return 401 unauthenticated, 403 authenticated but not permitted, 404 for a missing record or a record whose file is gone, as agreed with the Backend Lead. - Cap the EOI upload at 25MB, matching the disk uploads. It had no limit and buffered whole files in memory. Also check the content type as well as the file extension. - Map upload rejections to 400 instead of 500, with the size limit in the message. - Point incident attachment fileUrl at the route that serves the file. It pointed at /uploads/<filename>, which nothing serves. - Fix a query that passed a string where an ObjectId was required. "documents" is on the Guard discriminator, not the base User schema, so Mongoose could not cast it and the lookup never matched. This also unbroke two existing admin endpoints. - Align the guard registration Swagger with the implemented behaviour (25MB, images plus PDF, video and audio) rather than tightening the code. - Add 16 tests covering path resolution, authorisation, missing files and traversal attempts.
- Mount handleUploadError on the incident attachment upload route. A rejected file type already returned 400 because the filter sets a status, but an oversized file surfaced as a MulterError with no status and reached the global handler as a 500. Both now return 400 with the reason. - Rework the test file to match how the rest of the suite mocks. It was the only test mocking node built-ins, and the partial fs mock would break if the import chain ever pulled in a module that touches the filesystem at load time. It now mocks the project's own uploadsDir module to point at a temp folder and uses the real filesystem. - Add four cases for handleUploadError: an oversized file returns 400 stating the limit, any other multer error returns 400 rather than 500, a rejected file type carries its reason, and an unrelated failure is passed on instead of being reported as a bad request.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements BE 028, Make Uploaded Licence and Incident Files Retrievable and Persistent.
Uploaded files were written to disk but there was no way to fetch them back, and they were lost whenever the backend container was rebuilt. One upload path also had no size limit at all.
All seven checklist items on the ticket are covered.
What changed
Persistence. Added a named docker volume for the uploads directory so files survive container rebuilds. Previously the container filesystem was discarded on rebuild and every uploaded file went with it.
Retrieval. Added
GET /api/v1/documents/:id/file, which returns the file behind a document record, for example a guard's licence image.Safe path handling. Stored file references are resolved through
utils/uploadPath.js, which keeps the final path inside the uploads directory. A stored value containing../cannot reach other files on the server.Incident file URLs. Uploading an incident attachment returned
fileUrl: /uploads/<filename>, which nothing serves, so the URL always failed. It now points atGET /api/v1/incidents/{id}/attachments/{attachmentId}, which is the route that actually serves the file.Upload limits. The EOI upload had no size limit and buffered whole files in memory. It is now capped at 25MB to match the disk uploads, and checks the content type as well as the file extension, since an extension check alone passes anything renamed to
.pdf. The limit now lives in one exported constant so the two upload paths cannot drift apart again.Error codes. Upload rejections were reaching the global handler and being reported as 500. They now return 400 with the reason, including the size limit in the message.
Access rules
As agreed with @LoopyB, least privilege:
Status codes are 401 unauthenticated, 403 authenticated but not permitted, 404 for a missing record or a record whose file is gone.
Note this is deliberately different from the availability endpoints in #449, which return 404 rather than 403 to avoid confirming a record exists. 403 here follows the spec agreed for this ticket rather than that earlier pattern.
Decisions worth recording
Employer licence access is out of scope, as agreed. There is no reliable way to establish a guard to employer relationship in the current model.
Shiftoffersapplicants,acceptedByandguardIds, which mean applied for, accepted, and assigned. Those are three different things, and choosing which one grants access to somebody's licence document is an access policy decision rather than an implementation detail. Employer access to incident attachments is kept, becauseincident.shiftIdtoShift.createdByis an explicit single relationship and the existing route already enforces it.Guard registration Swagger discrepancy. The documentation said images only with a 5MB limit. The code accepts images, PDF, video and audio up to 25MB. On @LoopyB's direction I aligned the documentation to the implemented behaviour rather than tightening the code, since restricting the accepted types or size would be a behavioural change and belongs in its own ticket. The Swagger now describes what the endpoint actually does, and says as much.
Found while working on this
Three things outside the strict scope of the ticket. Happy to split any of them out if you would rather.
1. A query bug that had broken two existing endpoints.
getDocumentByIdand the expiry update both ranUser.findOne({ "documents._id": docId })with a string.documentsis declared on theGuarddiscriminator rather than the baseUserschema, so Mongoose cannot resolve the path to cast it and the query silently matched nothing. Both endpoints returned "Document not found" for documents that exist. Confirmed directly in the database, where the query matches by ObjectId and returns zero by string. Fixed with an explicit cast, andGET /documents/admin/documents/:idnow returns real data instead of 404.2. The EOI upload had no size limit, covered above. Included here because it was a memory risk rather than a correctness one.
3. Seeded role permissions are a stale subset of the code defaults. Not fixed in this PR, flagging only. The
Roledocuments in the database carry far fewer permissions thanDEFAULT_ROLE_PERMISSIONSinmiddleware/rbac.js:authorizePermissionsreads the database row first and only falls back to the code map when no row exists, so the smaller list wins. The effect is that in a seeded environment nobody exceptsuper_admincan use an incident permission gated route, including uploading an incident attachment. I hit this while testing. It looks like the same class of role definition drift already noted in the architecture doc.Testing
Unit tests. 16 new tests in
tests/documentFile.controller.test.js, covering path resolution and every access rule.Full suite compared against
mainwith and without this branch, to confirm nothing regressed:Same eleven pre-existing suite failures on both, plus this branch's suite and its 16 tests. Lint and
format:checkare clean.Live checks against the running stack, authenticated as the seeded guard, a second guard, and an admin:
Incident URL, end to end. Uploaded an attachment, took the returned URL from the response and requested it:
Persistence. Wrote a file into the uploads volume, destroyed the backend container with
docker compose rm -sf backend, recreated it, and the file was still there.Notes
main. There was a conflict inconfig/multer.jsbecause the uploads directory was moved to<backend root>/uploadsand theimport.meta.urlusage removed. I took that location and kept it in one shared module so writing and reading cannot drift apart. All checks above were re-run after the rebase.fileUrlvalue, which previously pointed at a path nothing serves and is not read by either client.