Navigating the Node.js 18/20 Upgrade: A Tale of Constants, JSON, and TypeScript Headaches

The end of December saw me diving into the exciting world of Node.js 18/20, upgrading all my Lambda functions to take advantage of these shiny new versions. It was smooth sailing for most, a quick and efficient process. However, one particularly complex and sensitive project, a labor of love spanning several years, presented a formidable challenge. With the deadline looming, I found myself in a frantic scramble, converting and debugging my Lambda functions and layers until the very last minute.

While the majority of the changes involved simple cosmetic adjustments, a few thorny issues arose from the way I was using modules. One such problem stemmed from the fact that my CDK project was written in TypeScript, my Lambda functions and layers were in JavaScript, and I had certain resources that needed direct access from both.

This brings us to the heart of the matter: importing JSON across different JavaScript flavors. Let's explore the challenges and solutions I encountered.

The Dilemma: A Single Source of Truth for Constants

My goal was simple: I wanted to store all my application constants in a single, central JSON file, constants.json, and access them seamlessly from my Lambda layers, functions, and CDK definitions. This would ensure consistency and simplify maintenance.

The Initial Approach: A JavaScript Module for Constants

Initially, I opted for a straightforward solution: I created a JavaScript module, constants.js, to hold my constants. This module could be easily required by both my Lambda functions and CDK definitions.

// layers/src/utility-layer/utility-layer/constants.js
const constants = {
  CONSTANT_SET: {
    CONSTANT_1: "abc",
    CONSTANT_2: "def",
    // ... more constants
  },
  // ... more constant sets
};
module.exports = constants;

This solution was functional, but it felt somewhat clunky. My TypeScript code used require to access the constants, which felt somewhat out of place in a TypeScript environment. I yearned for a more elegant, TypeScript-native solution.

The TypeScript Solution: `import` Assertions and .mjs Files

TypeScript introduced the import assertion feature, allowing direct import of JSON files:

// lib/myresource.ts
import constants from "./something.json" assert { type: "json" };

However, this feature is not supported in ES modules, a crucial limitation for my Lambda environment. This left me searching for alternative solutions.

Furthermore, importing directly from .mjs modules without a declaration file was a headache my TypeScript code wasn't willing to endure. To bypass this limitation, I embarked on a journey to find a compromise that satisfied both my TypeScript and JavaScript needs.

The Hybrid Solution: Constants.json and Constants.mjs

The solution I ultimately arrived at involved a combination of the constants.json file and a wrapper constants.mjs file. This hybrid approach offered the best of both worlds:

constants.json: The single source of truth for all constants, residing within my Lambda layer.
constants.mjs: A wrapper module that reads the constants.json file and exports its contents.

// layers/src/utility-layer/utility-layer/constants.json
{
  "CONSTANT_SET": {
    "CONSTANT_1": "abc",
    "CONSTANT_2": "def",
    // ... more constants
  },
  // ... more constant sets
}

// layers/src/utility-layer/utility-layer/constants.mjs
import fs from "fs";
export default JSON.parse(fs.readFileSync("/opt/nodejs/utility-layer/constants.json", "utf8"));

With this setup, I could import the constants in my Lambda functions and layers:

// javascript_file.mjs
import constants from '/opt/nodejs/utility-layer/constants.mjs';

And in my CDK TypeScript definitions:

// lib/myresource.ts
import constants from '../layers/src/utility-layer/utility-layer/constants.json';

This hybrid approach offered a clean solution:

Consistency: constants.json served as the single source of truth, ensuring consistent access to constants across my application.
TypeScript Integration: The ability to directly import constants.json in TypeScript provided a more native experience.
JavaScript Compatibility: The constants.mjs wrapper ensured compatibility with my JavaScript Lambda functions and layers.

Performance Considerations

While this approach provided a workable solution, I acknowledge that it might not be the most performant. Reading the JSON file from disk every time a constant is required could potentially introduce latency.

For performance-critical applications, other options such as storing constants in memory or utilizing more efficient file access methods might be worth considering. However, for my use case, this solution strikes a good balance between simplicity and efficiency.

Conclusion: A Triumph of (Almost) Seamless Integration

The Node.js 18/20 upgrade, despite its initial challenges, ultimately proved a successful endeavor. The journey involved navigating the nuances of TypeScript and JavaScript compatibility, leading to a hybrid solution that allowed me to access my constants seamlessly across different parts of my application.

While I readily admit that this solution might not be the most elegant or performant, it was efficient enough to meet my requirements. It served as a valuable lesson in understanding the complexities of working with different JavaScript flavors within a single project, highlighting the importance of flexible and adaptable solutions.

In the end, I can proudly declare: "Done!" The upgrade is complete, and I stand ready to embrace the possibilities of these latest Node.js versions.