Как в цикле перебрать содержимое файлов на NodeJS?

Question

[Александр Дегтярев]

Добрый вечер!

Есть несколько json файлов, нужно с помощью NodeJS открывать эти файлы, брать информацию, записывать накопительно в переменную и в итоге отправлять содержимое на клиент. Дело в том что не нужно блокировать основной поток и использовать только асинхронные методы.

Answer 1

[Михаил Р.]

использовать только асинхронные методы

fs/promises:
1. Получить список файлов директории - fs.readdir.
1.1. Отфильтровать файлы только с расширением .json.
2. Чтение файлов в цикле - fs.readFile.

и в итоге отправлять содержимое на клиент

Здесь лучше начать с запроса от клиента на сервер.

Comments

[Александр Дегтярев]

Благодарю. Возможно ли достигнуть того же варианта с использованием callback-ов?

[Михаил Р.]

Александр Дегтярев, возможно, но "callback hell" никто не отменял...

[Yuriy Vorobyov]

Александр Дегтярев, почему нет? в Callback API все методы есть

[Александр Дегтярев]

Yuriy Vorobyov, не выходит, не выдает итоговый набор данных. Если один файл то нормально, а здесь может быть разное количество файлов которые нужно пройти открыв каждый и вытащив данные. С промисами тоже нормально, я хочу понять как это можно сделать при помощи функций обратного вызова

[Yuriy Vorobyov]

Александр Дегтярев, если честно, не понимаю проблему

есть файлы json:

package-lock.json

{
  "name": "nodejs",
  "version": "1.0.0",
  "lockfileVersion": 2,
  "requires": true,
  "packages": {
    "": {
      "name": "nodejs",
      "version": "1.0.0",
      "license": "ISC",
      "dependencies": {
        "@types/node": "^18.0.6",
        "node-fetch": "^3.2.6"
      }
    },
    "node_modules/@types/node": {
      "version": "18.0.6",
      "resolved": "https://registry.npmjs.org/@types/node/-/node-18.0.6.tgz",
      "integrity": "sha512-/xUq6H2aQm261exT6iZTMifUySEt4GR5KX8eYyY+C4MSNPqSh9oNIP7tz2GLKTlFaiBbgZNxffoR3CVRG+cljw=="
    },
    "node_modules/data-uri-to-buffer": {
      "version": "4.0.0",
      "resolved": "https://registry.npmjs.org/data-uri-to-buffer/-/data-uri-to-buffer-4.0.0.tgz",
      "integrity": "sha512-Vr3mLBA8qWmcuschSLAOogKgQ/Jwxulv3RNE4FXnYWRGujzrRWQI4m12fQqRkwX06C0KanhLr4hK+GydchZsaA==",
      "engines": {
        "node": ">= 12"
      }
    },
    "node_modules/fetch-blob": {
      "version": "3.1.5",
      "resolved": "https://registry.npmjs.org/fetch-blob/-/fetch-blob-3.1.5.tgz",
      "integrity": "sha512-N64ZpKqoLejlrwkIAnb9iLSA3Vx/kjgzpcDhygcqJ2KKjky8nCgUQ+dzXtbrLaWZGZNmNfQTsiQ0weZ1svglHg==",
      "funding": [
        {
          "type": "github",
          "url": "https://github.com/sponsors/jimmywarting"
        },
        {
          "type": "paypal",
          "url": "https://paypal.me/jimmywarting"
        }
      ],
      "dependencies": {
        "node-domexception": "^1.0.0",
        "web-streams-polyfill": "^3.0.3"
      },
      "engines": {
        "node": "^12.20 || >= 14.13"
      }
    },
    "node_modules/formdata-polyfill": {
      "version": "4.0.10",
      "resolved": "https://registry.npmjs.org/formdata-polyfill/-/formdata-polyfill-4.0.10.tgz",
      "integrity": "sha512-buewHzMvYL29jdeQTVILecSaZKnt/RJWjoZCF5OW60Z67/GmSLBkOFM7qh1PI3zFNtJbaZL5eQu1vLfazOwj4g==",
      "dependencies": {
        "fetch-blob": "^3.1.2"
      },
      "engines": {
        "node": ">=12.20.0"
      }
    },
    "node_modules/node-domexception": {
      "version": "1.0.0",
      "resolved": "https://registry.npmjs.org/node-domexception/-/node-domexception-1.0.0.tgz",
      "integrity": "sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ==",
      "funding": [
        {
          "type": "github",
          "url": "https://github.com/sponsors/jimmywarting"
        },
        {
          "type": "github",
          "url": "https://paypal.me/jimmywarting"
        }
      ],
      "engines": {
        "node": ">=10.5.0"
      }
    },
    "node_modules/node-fetch": {
      "version": "3.3.0",
      "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-3.3.0.tgz",
      "integrity": "sha512-BKwRP/O0UvoMKp7GNdwPlObhYGB5DQqwhEDQlNKuoqwVYSxkSZCSbHjnFFmUEtwSKRPU4kNK8PbDYYitwaE3QA==",
      "dependencies": {
        "data-uri-to-buffer": "^4.0.0",
        "fetch-blob": "^3.1.4",
        "formdata-polyfill": "^4.0.10"
      },
      "engines": {
        "node": "^12.20.0 || ^14.13.1 || >=16.0.0"
      },
      "funding": {
        "type": "opencollective",
        "url": "https://opencollective.com/node-fetch"
      }
    },
    "node_modules/web-streams-polyfill": {
      "version": "3.2.1",
      "resolved": "https://registry.npmjs.org/web-streams-polyfill/-/web-streams-polyfill-3.2.1.tgz",
      "integrity": "sha512-e0MO3wdXWKrLbL0DgGnUV7WHVuw9OUvL4hjgnPkIeEvESk74gAITi5G606JtZPp39cd8HA9VQzCIvA49LpPN5Q==",
      "engines": {
        "node": ">= 8"
      }
    }
  },
  "dependencies": {
    "@types/node": {
      "version": "18.0.6",
      "resolved": "https://registry.npmjs.org/@types/node/-/node-18.0.6.tgz",
      "integrity": "sha512-/xUq6H2aQm261exT6iZTMifUySEt4GR5KX8eYyY+C4MSNPqSh9oNIP7tz2GLKTlFaiBbgZNxffoR3CVRG+cljw=="
    },
    "data-uri-to-buffer": {
      "version": "4.0.0",
      "resolved": "https://registry.npmjs.org/data-uri-to-buffer/-/data-uri-to-buffer-4.0.0.tgz",
      "integrity": "sha512-Vr3mLBA8qWmcuschSLAOogKgQ/Jwxulv3RNE4FXnYWRGujzrRWQI4m12fQqRkwX06C0KanhLr4hK+GydchZsaA=="
    },
    "fetch-blob": {
      "version": "3.1.5",
      "resolved": "https://registry.npmjs.org/fetch-blob/-/fetch-blob-3.1.5.tgz",
      "integrity": "sha512-N64ZpKqoLejlrwkIAnb9iLSA3Vx/kjgzpcDhygcqJ2KKjky8nCgUQ+dzXtbrLaWZGZNmNfQTsiQ0weZ1svglHg==",
      "requires": {
        "node-domexception": "^1.0.0",
        "web-streams-polyfill": "^3.0.3"
      }
    },
    "formdata-polyfill": {
      "version": "4.0.10",
      "resolved": "https://registry.npmjs.org/formdata-polyfill/-/formdata-polyfill-4.0.10.tgz",
      "integrity": "sha512-buewHzMvYL29jdeQTVILecSaZKnt/RJWjoZCF5OW60Z67/GmSLBkOFM7qh1PI3zFNtJbaZL5eQu1vLfazOwj4g==",
      "requires": {
        "fetch-blob": "^3.1.2"
      }
    },
    "node-domexception": {
      "version": "1.0.0",
      "resolved": "https://registry.npmjs.org/node-domexception/-/node-domexception-1.0.0.tgz",
      "integrity": "sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ=="
    },
    "node-fetch": {
      "version": "3.3.0",
      "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-3.3.0.tgz",
      "integrity": "sha512-BKwRP/O0UvoMKp7GNdwPlObhYGB5DQqwhEDQlNKuoqwVYSxkSZCSbHjnFFmUEtwSKRPU4kNK8PbDYYitwaE3QA==",
      "requires": {
        "data-uri-to-buffer": "^4.0.0",
        "fetch-blob": "^3.1.4",
        "formdata-polyfill": "^4.0.10"
      }
    },
    "web-streams-polyfill": {
      "version": "3.2.1",
      "resolved": "https://registry.npmjs.org/web-streams-polyfill/-/web-streams-polyfill-3.2.1.tgz",
      "integrity": "sha512-e0MO3wdXWKrLbL0DgGnUV7WHVuw9OUvL4hjgnPkIeEvESk74gAITi5G606JtZPp39cd8HA9VQzCIvA49LpPN5Q=="
    }
  }
}

package.json

{
  "name": "nodejs",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@types/node": "^18.0.6",
    "node-fetch": "^3.2.6"
  }
}

Можно написать вот такой вот код:

const fs = require('node:fs');
const path = require('path')

const result = '';

fs.readdir(__dirname, (err, files) => {
  const jsonFiles = files.filter(file => path.extname(file) === '.json');

  jsonFiles.map(file => fs.readFile(file, 'utf8', (err, data) => {
    result += data;
  })) 
})

[Yuriy Vorobyov]

Результат:

result

{
  "name": "nodejs",
  "version": "1.0.0",
  "lockfileVersion": 2,
  "requires": true,
  "packages": {
    "": {
      "name": "nodejs",
      "version": "1.0.0",
      "license": "ISC",
      "dependencies": {
        "@types/node": "^18.0.6",
        "node-fetch": "^3.2.6"
      }
    },
    "node_modules/@types/node": {
      "version": "18.0.6",
      "resolved": "https://registry.npmjs.org/@types/node/-/node-18.0.6.tgz",
      "integrity": "sha512-/xUq6H2aQm261exT6iZTMifUySEt4GR5KX8eYyY+C4MSNPqSh9oNIP7tz2GLKTlFaiBbgZNxffoR3CVRG+cljw=="
    },
    "node_modules/data-uri-to-buffer": {
      "version": "4.0.0",
      "resolved": "https://registry.npmjs.org/data-uri-to-buffer/-/data-uri-to-buffer-4.0.0.tgz",
      "integrity": "sha512-Vr3mLBA8qWmcuschSLAOogKgQ/Jwxulv3RNE4FXnYWRGujzrRWQI4m12fQqRkwX06C0KanhLr4hK+GydchZsaA==",
      "engines": {
        "node": ">= 12"
      }
    },
    "node_modules/fetch-blob": {
      "version": "3.1.5",
      "resolved": "https://registry.npmjs.org/fetch-blob/-/fetch-blob-3.1.5.tgz",
      "integrity": "sha512-N64ZpKqoLejlrwkIAnb9iLSA3Vx/kjgzpcDhygcqJ2KKjky8nCgUQ+dzXtbrLaWZGZNmNfQTsiQ0weZ1svglHg==",
      "funding": [
        {
          "type": "github",
          "url": "https://github.com/sponsors/jimmywarting"
        },
        {
          "type": "paypal",
          "url": "https://paypal.me/jimmywarting"
        }
      ],
      "dependencies": {
        "node-domexception": "^1.0.0",
        "web-streams-polyfill": "^3.0.3"
      },
      "engines": {
        "node": "^12.20 || >= 14.13"
      }
    },
    "node_modules/formdata-polyfill": {
      "version": "4.0.10",
      "resolved": "https://registry.npmjs.org/formdata-polyfill/-/formdata-polyfill-4.0.10.tgz",
      "integrity": "sha512-buewHzMvYL29jdeQTVILecSaZKnt/RJWjoZCF5OW60Z67/GmSLBkOFM7qh1PI3zFNtJbaZL5eQu1vLfazOwj4g==",
      "dependencies": {
        "fetch-blob": "^3.1.2"
      },
      "engines": {
        "node": ">=12.20.0"
      }
    },
    "node_modules/node-domexception": {
      "version": "1.0.0",
      "resolved": "https://registry.npmjs.org/node-domexception/-/node-domexception-1.0.0.tgz",
      "integrity": "sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ==",
      "funding": [
        {
          "type": "github",
          "url": "https://github.com/sponsors/jimmywarting"
        },
        {
          "type": "github",
          "url": "https://paypal.me/jimmywarting"
        }
      ],
      "engines": {
        "node": ">=10.5.0"
      }
    },
    "node_modules/node-fetch": {
      "version": "3.3.0",
      "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-3.3.0.tgz",
      "integrity": "sha512-BKwRP/O0UvoMKp7GNdwPlObhYGB5DQqwhEDQlNKuoqwVYSxkSZCSbHjnFFmUEtwSKRPU4kNK8PbDYYitwaE3QA==",
      "dependencies": {
        "data-uri-to-buffer": "^4.0.0",
        "fetch-blob": "^3.1.4",
        "formdata-polyfill": "^4.0.10"
      },
      "engines": {
        "node": "^12.20.0 || ^14.13.1 || >=16.0.0"
      },
      "funding": {
        "type": "opencollective",
        "url": "https://opencollective.com/node-fetch"
      }
    },
    "node_modules/web-streams-polyfill": {
      "version": "3.2.1",
      "resolved": "https://registry.npmjs.org/web-streams-polyfill/-/web-streams-polyfill-3.2.1.tgz",
      "integrity": "sha512-e0MO3wdXWKrLbL0DgGnUV7WHVuw9OUvL4hjgnPkIeEvESk74gAITi5G606JtZPp39cd8HA9VQzCIvA49LpPN5Q==",
      "engines": {
        "node": ">= 8"
      }
    }
  },
  "dependencies": {
    "@types/node": {
      "version": "18.0.6",
      "resolved": "https://registry.npmjs.org/@types/node/-/node-18.0.6.tgz",
      "integrity": "sha512-/xUq6H2aQm261exT6iZTMifUySEt4GR5KX8eYyY+C4MSNPqSh9oNIP7tz2GLKTlFaiBbgZNxffoR3CVRG+cljw=="
    },
    "data-uri-to-buffer": {
      "version": "4.0.0",
      "resolved": "https://registry.npmjs.org/data-uri-to-buffer/-/data-uri-to-buffer-4.0.0.tgz",
      "integrity": "sha512-Vr3mLBA8qWmcuschSLAOogKgQ/Jwxulv3RNE4FXnYWRGujzrRWQI4m12fQqRkwX06C0KanhLr4hK+GydchZsaA=="
    },
    "fetch-blob": {
      "version": "3.1.5",
      "resolved": "https://registry.npmjs.org/fetch-blob/-/fetch-blob-3.1.5.tgz",
      "integrity": "sha512-N64ZpKqoLejlrwkIAnb9iLSA3Vx/kjgzpcDhygcqJ2KKjky8nCgUQ+dzXtbrLaWZGZNmNfQTsiQ0weZ1svglHg==",
      "requires": {
        "node-domexception": "^1.0.0",
        "web-streams-polyfill": "^3.0.3"
      }
    },
    "formdata-polyfill": {
      "version": "4.0.10",
      "resolved": "https://registry.npmjs.org/formdata-polyfill/-/formdata-polyfill-4.0.10.tgz",
      "integrity": "sha512-buewHzMvYL29jdeQTVILecSaZKnt/RJWjoZCF5OW60Z67/GmSLBkOFM7qh1PI3zFNtJbaZL5eQu1vLfazOwj4g==",
      "requires": {
        "fetch-blob": "^3.1.2"
      }
    },
    "node-domexception": {
      "version": "1.0.0",
      "resolved": "https://registry.npmjs.org/node-domexception/-/node-domexception-1.0.0.tgz",
      "integrity": "sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ=="
    },
    "node-fetch": {
      "version": "3.3.0",
      "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-3.3.0.tgz",
      "integrity": "sha512-BKwRP/O0UvoMKp7GNdwPlObhYGB5DQqwhEDQlNKuoqwVYSxkSZCSbHjnFFmUEtwSKRPU4kNK8PbDYYitwaE3QA==",
      "requires": {
        "data-uri-to-buffer": "^4.0.0",
        "fetch-blob": "^3.1.4",
        "formdata-polyfill": "^4.0.10"
      }
    },
    "web-streams-polyfill": {
      "version": "3.2.1",
      "resolved": "https://registry.npmjs.org/web-streams-polyfill/-/web-streams-polyfill-3.2.1.tgz",
      "integrity": "sha512-e0MO3wdXWKrLbL0DgGnUV7WHVuw9OUvL4hjgnPkIeEvESk74gAITi5G606JtZPp39cd8HA9VQzCIvA49LpPN5Q=="
    }
  }
}

{
  "name": "nodejs",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@types/node": "^18.0.6",
    "node-fetch": "^3.2.6"
  }
}

[Александр Дегтярев]

Yuriy Vorobyov, всегда думал что из за синхронности так нельзя сделать, делал примерно такое же но перебирал for и у меня result всегда был undefined. Благодарю за решение, буду смотреть.

[Yuriy Vorobyov]

Александр Дегтярев,

из за синхронности так нельзя сделать

- что вы под этим подразумеваете?