Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

url cleanup upgrade #148

Open
wants to merge 14 commits into
base: release/0.2.0
Choose a base branch
from
Prev Previous commit
Next Next commit
CR fixes
  • Loading branch information
Shay.Gazit authored and Shay.Gazit committed Mar 13, 2021
commit 3720c2eec923323e230036e23b90c553ba56dc6d
2 changes: 2 additions & 0 deletions .env.development
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ BASE_REDIRECT_URL=http:https://localhost:3000/u/
LOG_LEVEL=debug
npm_package_type=module

URL_EXPIRE_FROM=create #update
shay123g marked this conversation as resolved.
Show resolved Hide resolved

########## Storage ##########
## InMemory/Relational
STORAGE_DRIVER=InMemory
Expand Down
1 change: 1 addition & 0 deletions src/config/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ const rawConfig: RawConfig = {
url: {
matchPattern: process.env.URL_MATCH_PATTERN || '**',
lifetime: process.env.URL_LIFETIME || '7 days',
urlExpireFrom: process.env.URL_EXPIRE_FROM || 'creationTime',
},
storage: {
driverName: process.env.STORAGE_DRIVER || '',
Expand Down
1 change: 1 addition & 0 deletions src/config/normalize.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ export function normalizeConfig({
baseRedirectUrl,
url: {
lifetimeMs: ms(url.lifetime),
urlExpireFrom : url.urlExpireFrom,
matchPattern: url.matchPattern,
cleanupIntervalMs,
},
Expand Down
2 changes: 2 additions & 0 deletions src/config/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ export interface RawConfig {
url: {
matchPattern: string
lifetime: string
urlExpireFrom: string
}
storage: {
driverName: string
Expand Down Expand Up @@ -39,6 +40,7 @@ export interface Config {
url: {
matchPattern: string
lifetimeMs: number
urlExpireFrom: string
cleanupIntervalMs: number
}
baseRedirectUrl: string
Expand Down
17 changes: 15 additions & 2 deletions src/services/storage/drivers/inMemory/index.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,17 @@
import cryptoRandomString from 'crypto-random-string'
import { NotFoundError } from '../../../../errors/notFound.js'
import { InvalidConfigError } from '../../../../errors/invalidConfig.js'
import { InMemoryStorageConfig } from '../../types/config.js'
import type { StorageDriver } from '../../types/index.js'
import type { StoredUrl, UrlWithInformation, UrlRequestData } from '../../types/url.js'
import { getRawConfig } from '../../../../config/__test__/helpers'
import { logger } from '../../../logger/logger'

function validateUrlExpireFrom(deleteFrom: string) {
if (deleteFrom !== 'create' && deleteFrom !== 'update') {
throw new InvalidConfigError("URL_EXPIRE_FROM value must be 'create' or 'update'")
}
}

export class InMemoryStorage implements StorageDriver {
data: { urls: Map<string, StoredUrl>; urlInformation: Map<string, UrlWithInformation> } = {
Expand Down Expand Up @@ -39,12 +48,16 @@ export class InMemoryStorage implements StorageDriver {
this.storage.data.urls.delete(id)
}
public async deleteOverdue(timespanMs: number): Promise<number> {
const deleteFrom = getRawConfig().url.urlExpireFrom
logger.debug('urlExpireFrom is {}', deleteFrom)
validateUrlExpireFrom(deleteFrom)
const deleteBefore = new Date().getTime() - timespanMs
let deletedCount = 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the comment in the PR


this.storage.data.urls.forEach((storedUrl) => {
const createdAt = new Date(storedUrl.createdAt).getTime()
if (createdAt <= deleteBefore) {
let relativeDate = new Date(storedUrl.createdAt).getTime()
if (deleteFrom === 'update') relativeDate = new Date(storedUrl.updatedAt).getTime()
shay123g marked this conversation as resolved.
Show resolved Hide resolved
if (relativeDate <= deleteBefore) {
this.storage.data.urls.delete(storedUrl.id)
deletedCount++
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where's the Relational implementation?

Expand Down