Disclaimer: да, я знаю, что бест-практики рекомендуют 4 ноды. но в наличии есть только 3 ноды с идентичной версией, которых запускаю с 4 "дисками" (на самом деле просто папки на одном raid-массиве), чтобы заработал erasure-coding (мне он не нужен, т.к резервинованием по диску занимается слой ниже) но, как я понял, distributed mode без него не работает
/etc/defaults/minio
MINIO_VOLUMES="https://node1:9443/home/minio/data/1 https://node1:9443/home/minio/data/2 https://node1:9443/home/minio/data/3 https://node1:9443/home/minio/data/4 https://node2:9443/home/minio/data/1 https://node2:9443/home/minio/data/2 https://node2:9443/home/minio/data/3 https://node2:9443/home/minio/data/4 https://node2:9443/home/minio/data/5 https://node2:9443/home/minio/data/6 https://node3:9443/home/minio/data/1 https://node3:9443/home/minio/data/2 https://node3:9443/home/minio/data/3 https://node3:9443/home/minio/data/4 https://node3:9443/home/minio/data/5 https://node3:9443/home/minio/data/6"
MINIO_OPTS="--address myipv4:9443 --console-address myipv4:9443"
MINIO_ROOT_USER=someuser
MINIO_ROOT_PASSWORD=somepassword
MINIO_ACCESS_KEY=testkey
MINIO_SECRET_KEY=testpass
MINIO_SERVER_URL="https://real-load-balancer-uri"
вызов из systemd
ExecStart=/home/minio/bin/minio server $MINIO_OPTS $MINIO_VOLUMES
при старте в журнале спам:
Dec 23 22:12:15 s-3-1 minio[90816]: Unable to read 'format.json' from https://node1:9443/home/minio/data/1: Unexpected RPC call at this path '/minio/storage/home/minio/data/1/v42/readall' - *rolling upgrade is not allowed* - please make sure all servers are running thesame MinIO version (RELEASE.2021-12-20T22-07-16Z)
Dec 23 22:12:15 s-3-1 minio[90816]: Unable to read 'format.json' from https://node2:9443/home/minio/data/1: Unexpected RPC call at this path '/minio/storage/home/minio/data/1/v42/readall' - *rolling upgrade is not allowed* - please make sure all servers are running thesame MinIO version (RELEASE.2021-12-20T22-07-16Z)
Dec 23 22:12:15 s-3-1 minio[90816]: Unable to read 'format.json' from https://node3:9443/home/minio/data/1: Unexpected RPC call at this path '/minio/storage/home/minio/data/1/v42/readall' - *rolling upgrade is not allowed* - please make sure all servers are running thesame MinIO version (RELEASE.2021-12-20T22-07-16Z)
Dec 23 22:12:15 s-3-1 minio[90816]: API: SYSTEM()
Dec 23 22:12:15 s-3-1 minio[90816]: Time: 22:12:15 MSK 12/23/2021Dec 23 22:12:15 s-3-1 minio[90816]: Error: Read failed. Insufficient number of disks online (*errors.errorString)Dec 23 22:12:15
s-3-1 minio[90816]: 5: cmd/prepare-storage.go:272:cmd.connectLoadInitFormats()
Dec 23 22:12:15 storage-1 minio[90816]:4: cmd/prepare-storage.go:321:cmd.waitForFormatErasure()D
ec 23 22:12:15 storage-1 minio[90816]: 3: cmd/erasure-server-pool.go:91:cmd.newErasureServerPools()
Dec 23 22:12:15 storage-1 minio[90816]: 2: cmd/server-main.go:648:cmd.newObjectLayer()
Dec 23 22:12:15 storage-1 minio[90816]: 1: cmd/server-main.go:527:cmd.serverMain()
Dec 23 22:12:15 storage-1 minio[90816]: Waiting for a minimum of 6 disks to come online (elapsed 1m40s)
если дёргать адрес курлом - в ответ прилетает xml
<Error>
<Code>XMinioServerNotInitialized</Code>
<Message>Server not initialized, please try again.</Message>
<Key>minio/data/3</Key>
<BucketName>home</BucketName>
<Resource>/home/minio/data/3</Resource>
<RequestId>16C3785B916902E7</RequestId>
<HostId/>
</Error>
как правильно запуститься в такой схеме?
Простой