DCC Scanner
DCC Scanner is a full-stack application built for scanning and extracting DCC and R&R data from websites.
URLs:
- Azure: https://dcc-scanner.azurewebsites.net
- Mogenius: https://dcc-scanner-prod-projects-f0597n.mo5.mogenius.io
Frontend is built using Next JS. Styling is handled using Tailwind. Global state is handled using Easy Peasy state manager.
Backend is created using Node and Express.
Architecture setup
DCC Scanner consists of a single server-side rendered application instance.
Frontend
User interface routes are built using Next JS and its getServerSideProps functionality.
Backend
API routes are handled by custom Node JS Express server. Separate instances are hosted on Azure and Mogenius for redundancy. Database used in the backend is Redis hosted on Redis Cloud.
Frontend and backend communicate via http/1.1, Server-Sent Events and polling.
Backend is fetching the external website's data needed for extracting DCC and R&R.
DCC Scanner Architecture diagram
Pipeline setup
Frontend and backend pipelines consist of a single instance. Same setup is deployed on Azure and Mogenius using different tools.
Azure
Code is hosted in Azure DevOps Repo. Pushing to master branch triggers the production Azure DevOps pipeline that is using a Self-hosted agent for building and pushing the Image to DockerHub Container Repository. Pipeline is deploying the image to Azure App Service for Containers on Linux.
DCC Scanner Pipeline diagram - Azure
Mogenius
Code is hosted in GitHub Repo. Pushing to master branch triggers the integration between GitHub and Mogenius. Mogenius pulls the code from GitHub, detects the Dockerfile, and builds and deploys the image.
DCC Scanner Pipeline diagram - Mogenius
Project Details
Application consists of the following screens:
- Login
- Main Page
- New DCC Scan
- R&R Scan
Login screen - DCC Scanner is using self-hosted authentication.
Main page screen - Scanned page metadata and completed scan results are stored in Indexed DB and displayed in cards as thumbnails. Scan results can be viewed in a spreadsheet form and downloaded in .xlsx, .csv and .json formats. Multiple scans of a single website are stored under the same ID and can be downloaded as a single .xlsx workbook where each separate scan is a single worksheet.
Adding the website's URL in the New DCC Scan field will fetch and display the website's metadata, and also fetch the website's XML sitemap. Sitemap will be parsed and individual page URL path/slug portions will be displayed in the Category drop-down. This is used to limit the scan to pages that belong to single Category. If multiple Categories are selected, all pages from selected Categories are scanned.
For New DCC Scan, App will parse the URLs and scan each one individually. Each step of the scan will be sent to the fronted in form of Server-Sent Events (SSE) and displayed in the Log section. Once the scan is completed, the website's metadata and scan results are stored in Indexed DB. For subsequent scans of the same website, the metadata is fetched from the Indexed DB.
New R&R Scan is started by selecting the Brand for scanning all available locales, or by selecting the Brand and Locale for scanning a single locale. DCC data will be extracted for each locale, and one test review submission will be performed on the server for extracting the R&R data. DCC and R&R data will be validated and the results will be displayed on the page. When scanning all available Locales, scan is performed for each locale sequentially and the results are fetched by using a standard polling method.