Skip to content

Technologies sort fix for the category overview #50

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,14 +56,15 @@ Lists available categories.
#### Categories Parameters

- `category` (optional): Filter by category name(s) - comma-separated list
- `client` (optional): Filter by client type (e.g., `mobile`, `desktop`) - defaults to `mobile`
- `onlyname` (optional): If present, returns only category names
- `fields` (optional): Comma-separated list of fields to include in the response (see [Field Selection API Documentation](#field-selection-api-documentation) for details)

#### Categories Response

```bash
curl --request GET \
--url 'https://d{{HOST}}/v1/categories?category=Domain%20parking%2CCI'
--url 'https://{{HOST}}/v1/categories?category=Domain%20parking%2CCI&client=desktop'
```

```json
Expand All @@ -74,10 +75,7 @@ curl --request GET \
"Jenkins",
"TeamCity"
],
"origins": {
"mobile": 22,
"desktop": 35
},
"origins": 22,
"category": "CI"
},
{
Expand All @@ -86,10 +84,7 @@ curl --request GET \
"Cloudflare",
"Arsys Domain Parking"
],
"origins": {
"mobile": 14,
"desktop": 8
},
"origins": 14,
"category": "Domain parking"
}
]
Expand Down Expand Up @@ -213,6 +208,7 @@ Provides technology adoption data.
- `rank` (required): Filter by rank
- `start` (optional): Filter by date range start (YYYY-MM-DD or 'latest')
- `end` (optional): Filter by date range end (YYYY-MM-DD)
- `version` (optional): Filter by version name(s) - comma-separated list (only when single technology is specified)

#### Adoption Response

Expand Down Expand Up @@ -248,6 +244,7 @@ Provides Core Web Vitals metrics for technologies.
- `rank` (required): Filter by rank
- `start` (optional): Filter by date range start (YYYY-MM-DD or 'latest')
- `end` (optional): Filter by date range end (YYYY-MM-DD)
- `version` (optional): Filter by version name(s) - comma-separated list (only when single technology is specified)

#### CWV Response

Expand Down Expand Up @@ -291,6 +288,7 @@ Provides Lighthouse scores for technologies.
- `rank` (required): Filter by rank
- `start` (optional): Filter by date range start (YYYY-MM-DD or 'latest')
- `end` (optional): Filter by date range end (YYYY-MM-DD)
- `version` (optional): Filter by version name(s) - comma-separated list (only when single technology is specified)

#### Lighthouse Response

Expand Down Expand Up @@ -338,6 +336,7 @@ Provides Page Weight metrics for technologies.
- `rank` (required): Filter by rank
- `start` (optional): Filter by date range start (YYYY-MM-DD or 'latest')
- `end` (optional): Filter by date range end (YYYY-MM-DD)
- `version` (optional): Filter by version name(s) - comma-separated list (only when single technology is specified)

#### Page Weight Response

Expand Down Expand Up @@ -399,6 +398,7 @@ Provides Lighthouse audits for technologies.
- `rank` (required): Filter by rank
- `start` (optional): Filter by date range start (YYYY-MM-DD or 'latest')
- `end` (optional): Filter by date range end (YYYY-MM-DD)
- `version` (optional): Filter by version name(s) - comma-separated list (only when single technology is specified)

#### Audits Response

Expand Down Expand Up @@ -588,7 +588,7 @@ The categories and technologies endpoints now support custom field selection, al
- `category` - Category name
- `description` - Category description
- `technologies` - Array of technology names in the category
- `origins` - Array of origin companies/organizations
- `origins` - Number of origins

Get only category names:

Expand All @@ -608,7 +608,7 @@ GET /v1/categories?fields=category,description
- `category` - Category name
- `description` - Technology description
- `icon` - Icon filename
- `origins` - Array of origin companies/organizations
- `origins` - Dictionary with origins per client

Get only technology names and categories:

Expand Down Expand Up @@ -643,7 +643,7 @@ GET /v1/technologies?category=JavaScript%20Frameworks&fields=technology,icon

- `technology` - Technology name
- `version` - Version name
- `origins` - Mobile and desktop origins
- `origins` - Dictionary with origins per client

Get only technology and version names:

Expand Down
6 changes: 6 additions & 0 deletions src/__tests__/routes.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -233,6 +233,12 @@ describe('API Routes', () => {
expect(Array.isArray(res.body)).toBe(true);
});

it('should return adoption data with version parameter', async () => {
const res = await request(app).get('/v1/adoption?technology=WordPress&geo=ALL&rank=ALL&start=latest&version=5.0');
expect(res.statusCode).toEqual(200);
expect(Array.isArray(res.body)).toBe(true);
});

it('should handle missing required parameters', async () => {
const res = await request(app).get('/v1/adoption');
expect(res.statusCode).toEqual(400);
Expand Down
14 changes: 11 additions & 3 deletions src/controllers/categoriesController.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ const listCategories = async (req, res) => {
const queryBuilder = async (params) => {
/*
// Validate parameters
const supportedParams = ['category', 'onlyname', 'fields'];
const supportedParams = ['category', 'onlyname', 'fields', 'client'];
const providedParams = Object.keys(params);
const unsupportedParams = providedParams.filter(param => !supportedParams.includes(param));

Expand All @@ -20,6 +20,7 @@ const listCategories = async (req, res) => {
*/

const isOnlyNames = params.onlyname || typeof params.onlyname === 'string';
const client = params.client || 'mobile'; // Default client if not provided
const hasCustomFields = params.fields && !isOnlyNames;

let query = firestore.collection('categories').orderBy('category', 'asc');
Expand All @@ -32,11 +33,16 @@ const listCategories = async (req, res) => {
}
}

// Apply client filter
if (client) {
query = query.where('client', '==', client);
}

// Apply field selection
if (isOnlyNames) {
query = query.select('category');
} else if (hasCustomFields) {
const requestedFields = params.fields.split(',').map(f => f.trim());
const requestedFields = params.fields.split(',').map(f => f.trim()) || ['category', 'description', 'technologies', 'origins'];
query = query.select(...requestedFields);
}

Expand All @@ -56,7 +62,9 @@ const listCategories = async (req, res) => {
// Include onlyname and fields in cache key calculation
const customCacheKeyData = {
onlyname: req.query.onlyname || false,
fields: req.query.fields
fields: req.query.fields,
category: req.query.category ? validateArrayParameter(req.query.category, 'category') : [],
client: req.query.client || 'mobile',
};

await executeQuery(req, res, 'categories', queryBuilder, dataProcessor, customCacheKeyData);
Expand Down
35 changes: 21 additions & 14 deletions src/controllers/reportController.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import { firestoreOld } from '../utils/db.js';
const firestore = firestoreOld;
import { firestore } from '../utils/db.js';

import {
REQUIRED_PARAMS,
Expand Down Expand Up @@ -55,7 +54,7 @@ const createReportController = (reportType) => {

/*
// Validate supported parameters
const supportedParams = ['technology', 'geo', 'rank', 'start', 'end'];
const supportedParams = ['technology', 'geo', 'rank', 'start', 'end', 'version'];
const providedParams = Object.keys(params);
const unsupportedParams = providedParams.filter(param => !supportedParams.includes(param));

Expand All @@ -78,8 +77,15 @@ const createReportController = (reportType) => {
return;
}

// Validate and process technology array
const techArray = validateArrayParameter(params.technology, 'technology');
// Validate and process technologies
const technologiesArray = validateArrayParameter(params.technology, 'technology');

// Validate and process versions
// Apply version filter with special handling for 'ALL' case
let versionsArray = ['ALL'];
if (technologiesArray.length === 1 && params.version) {
versionsArray = validateArrayParameter(params.version, 'version');
}

// Handle 'latest' date substitution
let startDate = params.start;
Expand All @@ -91,7 +97,8 @@ const createReportController = (reportType) => {
const queryFilters = {
geo: params.geo,
rank: params.rank,
technology: techArray,
technology: technologiesArray,
version: versionsArray,
startDate: startDate,
endDate: params.end
};
Expand All @@ -113,21 +120,21 @@ const createReportController = (reportType) => {
query = query.where('rank', '==', params.rank);

// Apply technology filter with batch processing
query = query.where('technology', 'in', techArray);
query = query.where('technology', 'in', technologiesArray);

// Apply version filter with special handling for 'ALL' case
if (params.version && techArray.length === 1) {
//query = query.where('version', '==', params.version); // TODO: Uncomment when migrating to a new data schema
} else {
//query = query.where('version', '==', 'ALL');
}
// Apply version filter with batch processing
query = query.where('version', 'in', versionsArray);

// Apply date filters
if (startDate) query = query.where('date', '>=', startDate);
if (params.end) query = query.where('date', '<=', params.end);

// Apply field projection to optimize query
query = query.select('date', 'technology', config.dataField);
const selectFields = ['date', 'technology', config.dataField];
if (!(versionsArray.length === 1 && versionsArray[0] === 'ALL')) {
selectFields.push('version');
}
query = query.select(...selectFields);

// Execute query
const snapshot = await query.get();
Expand Down
4 changes: 3 additions & 1 deletion src/controllers/technologiesController.js
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,9 @@ const listTechnologies = async (req, res) => {
// Include onlyname and fields in cache key calculation
const customCacheKeyData = {
onlyname: req.query.onlyname || false,
fields: req.query.fields
fields: req.query.fields,
technology: req.query.technology ? validateTechnologyArray(req.query.technology) : [],
category: req.query.category ? validateArrayParameter(req.query.category, 'category') : [],
};

await executeQuery(req, res, 'technologies', queryBuilder, dataProcessor, customCacheKeyData);
Expand Down
7 changes: 7 additions & 0 deletions terraform/dev/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,13 @@ paths:
responses:
200:
description: String
/v1/versions:
get:
summary: versions
operationId: getVersions
responses:
200:
description: String
/v1/audits:
get:
summary: audits
Expand Down
2 changes: 1 addition & 1 deletion terraform/dev/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ variable "environment" {
variable "project_database" {
type = string
description = "The database name"
default = "tech-report-api-prod" // TODO: Update this to the DEV database name
default = "tech-report-api-dev"
}

variable "google_service_account_cloud_functions" {
Expand Down
Loading