400, message='Got more than 8190 bytes (8495) when reading request header field Content-Security-Policy-Report-Only.'?

Question

[Maxwell012]

Отправляю запрос используя библиотеку aiohttp, прилетает ошибка:

400, message='Got more than 8190 bytes (8495) when reading request header field Content-Security-Policy-Report-Only.'

Можно ли как-то 'порциями' получать ответ, или изменить лимит?

Comments

[Сергей Горностаев]

Заголовок больше 8 килобайт - это что-то противоестественное.

[Maxwell012]

Сергей Горностаев, странно, все сайты нормально отрабатывают, этот по структуре такой же и выдает такую ошибку

[Maxwell012]

Сергей Горностаев, https://www.sparmax.no/robots.txt, этот сайт выдает ошибку, если можете посмотрите его

[Сергей Горностаев]

Maxwell012, там реально огромный заголовок:
Content-Security-Policy-Report-Only: font-src x.klarnacdn.net fonts.gstatic.com fonts.gstatic.com/s s.sparmax.no s.sparmax.dk s.sparmax.se s.maxkjop.no s.skjargaard.no *.zopim.com data: static.lipscore.com *.fontawesome.com *.gstatic.com *.yotpo.com *.googleapis.com 'self' 'unsafe-inline'; form-action secure.authorize.net test.authorize.net geostag.cardinalcommerce.com geo.cardinalcommerce.com 1eafstag.cardinalcommerce.com 1eaf.cardinalcommerce.com centinelapistag.cardinalcommerce.com centinelapi.cardinalcommerce.com *.amazon.com *.amazon.co.uk *.amazon.co.jp *.amazon.jp *.amazon.it *.amazon.fr *.amazon.es *.amazon.de *.yotpo.com 'self' 'unsafe-inline'; frame-ancestors 'self'; frame-src fast.amc.demdex.net secure.authorize.net test.authorize.net geostag.cardinalcommerce.com geo.cardinalcommerce.com 1eafstag.cardinalcommerce.com 1eaf.cardinalcommerce.com centinelapistag.cardinalcommerce.com centinelapi.cardinalcommerce.com www.paypal.com www.sandbox.paypal.com player.vimeo.com *.amazon.com *.amazon.co.uk *.amazon.co.jp *.amazon.jp *.amazon.it *.amazon.fr *.amazon.es *.amazon.de *.payments-amazon.com *.payments-amazon.co.uk *.payments-amazon.co.jp *.payments-amazon.jp *.payments-amazon.it *.payments-amazon.fr *.payments-amazon.es *.payments-amazon.de *.playground.klarna.com cdn.klarna.com www.google.com *.klarna.com *.criteo.com *.hotjar.com *.doubleclick.net widget.trustpilot.com *.dotdigital-pages.com *.dotdigital.com cdn.dnky.co webchat.dotdigital.com c.paypal.com checkout.paypal.com assets.braintreegateway.com pay.google.com *.cardinalcommerce.com *.trustpilot.com *.yotpo.com 'self' 'unsafe-inline'; img-src assets.adobedtm.com amcglobal.sc.omtrdc.net dpm.demdex.net cm.everesttech.net widgets.magentocommerce.com data: www.googleadservices.com www.google-analytics.com www.paypalobjects.com t.paypal.com *.ftcdn.net *.behance.net www.paypal.com fpdbs.paypal.com fpdbs.sandbox.paypal.com *.vimeocdn.com s.ytimg.com camo.githubusercontent.com d3sbl0c71oxeok.cloudfront.net dhkkzdfmpzvap.cloudfront.net d2bpzs5y44q6e0.cloudfront.net d37shgu97oizpd.cloudfront.net d1zlqll3enr74n.cloudfront.net d1jynp0fpwn93a.cloudfront.net d2cb3tokgpwh3v.cloudfront.net d1re8bfxx3pw6e.cloudfront.net d35u8xwkxs8vpe.cloudfront.net d13s9xffygp5o.cloudfront.net d388nbw0dwi1jm.cloudfront.net d11p2vtu3dppaw.cloudfront.net d3r89hiip86hka.cloudfront.net dc7snq0c8ipyk.cloudfront.net d5c7kvljggzso.cloudfront.net d2h8yg3ypfzua1.cloudfront.net d1b556x7apj5fb.cloudfront.net draz1ib3z71v2.cloudfront.net dr6hdp4s5yzfc.cloudfront.net d2bomicxw8p7ii.cloudfront.net d3aypcdgvjnnam.cloudfront.net d2a3iuf10348gy.cloudfront.net *.ssl-images-amazon.com *.ssl-images-amazon.co.uk *.ssl-images-amazon.co.jp *.ssl-images-amazon.jp *.ssl-images-amazon.it *.ssl-images-amazon.fr *.ssl-images-amazon.es *.ssl-images-amazon.de *.media-amazon.com *.media-amazon.co.uk *.media-amazon.co.jp *.media-amazon.jp *.media-amazon.it *.media-amazon.fr *.media-amazon.es *.media-amazon.de *.clerk.io cdn.klarna.com *.playground.klarnaevt.com www.googletagmanager.com ssl.gstatic.com www.gstatic.com googleads.g.doubleclick.net www.google.com s.sparmax.no www.tryggehandel.no *.trollweb.no google-analytics.com dev.visualwebsiteoptimizer.com google.com/ads www.facebook.com *.bing.com *.criteo.com *.criteo.net *.zdassets.com *.clarity.ms x.klarnacdn.net *.doubleclick.net raw.githubusercontent.com/vippsas *.sparmax.se *.sparmax.dk *.maxkjop.no *.skjaraard.no s.skjargaard.no googleadservices.com v2assets.zopim.io www.google.no/ads www.google.se/ads www.google.dk/ads www.google.fr/ads www.google.co.uk/ads www.google.uk/ads www.google.com/ads www.google.de/ads www.google.pl/ads www.google.lt/ads www.google.es/ads www.google.lv/ads www.google.ee/ads www.google.th/ads www.google.no/pagead www.google.se/pagead www.google.dk/pagead www.google.fr/pagead www.google.co.uk/pagead www.google.uk/pagead www.google.com/pagead www.google.de/pagead www.google.pl/pagead www.google.lt/pagead www.google.es/pagead www.google.lv/pagead www.google.ee/pagead www.google.th/pagead gtm.sparmax.no *.klarnaevt.com sparmax.wpcloud.trollweb.no www.sandbox.paypal.com b.stats.paypal.com dub.stats.paypal.com assets.braintreegateway.com c.paypal.com checkout.paypal.com *.yotpo.com 'self' 'unsafe-inline'; script-src assets.adobedtm.com secure.authorize.net test.authorize.net www.googleadservices.com www.google-analytics.com www.paypalobjects.com js.braintreegateway.com www.paypal.com geostag.cardinalcommerce.com 1eafstag.cardinalcommerce.com geoapi.cardinalcommerce.com 1eafapi.cardinalcommerce.com songbird.cardinalcommerce.com includestest.ccdc02.com www.sandbox.paypal.com t.paypal.com s.ytimg.com video.google.com vimeo.com www.vimeo.com *.vimeocdn.com www.youtube.com polyfill.io *.payments-amazon.com *.payments-amazon.co.uk *.payments-amazon.co.jp *.payments-amazon.jp *.payments-amazon.it *.payments-amazon.fr *.payments-amazon.es *.payments-amazon.de *.clerk.io *.algolia.net *.algolianet.com cdn.klarna.com x.klarnacdn.net *.playground.klarna.com tagmanager.google.com www.googletagmanager.com ssl.google-analytics.com www.google.com www.gstatic.com s.sparmax.no googletagmanager.com *.lipscore.com frankanddick.dev s.kk-resources.com *.criteo.com *.criteo.net dev.visualwebsiteoptimizer.com *.hotjar.com *.bing.com *.de17a.com *.facebook.net google-analytics.com *.adform.net *.sleeknote.com *.zdassets.com google.com gstatic.com *.trustpilot.com *.klarnaservices.com *.clarity.ms *.klarnacdn.net *.klarna.com *.doubleclick.net *.sparmax.se *.sparmax.dk *.maxkjop.no *.skjaraard.no s.skjargaard.no partner.googleadservices.com www.tryggehandel.no *.zopim.com widget-mediator.zopim.com static.lipscore.com widget.trustpilot.com invitejs.trustpilot.com eu-library.klarnaservices.com sleeknotecustomerscripts.sleeknote.com static.zdassets.com *.trackedlink.net *.trackedweb.net *.dotdigital-pages.com cdn.dnky.co api.comapi.com webchat.dotdigital.com assets.braintreegateway.com c.paypal.com pay.google.com api.braintreegateway.com api.sandbox.braintreegateway.com client-analytics.braintreegateway.com client-analytics.sandbox.braintreegateway.com *.yotpo.com 'self' 'unsafe-inline' 'unsafe-eval'; style-src getfirebug.com tagmanager.google.com fonts.googleapis.com https://*.sparmax.no *.trollweb.no *.lipscore.com *.klarnacdn.net https://*.sparmax.se https://*.sparmax.dk https://*.maxkjop.no https://*.skjargaard.no unsafe-inline sparmax.wpcloud.trollweb.no x.klarnacdn.net static.lipscore.com cdn.dnky.co webchat.dotdigital.com *.fontawesome.com *.trustpilot.com *.yotpo.com *.googleapis.com 'self' 'unsafe-inline'; object-src 'self' 'unsafe-inline'; media-src static.zdassets.com 'self' 'unsafe-inline'; manifest-src 'self' 'unsafe-inline'; connect-src dpm.demdex.net amcglobal.sc.omtrdc.net geostag.cardinalcommerce.com geo.cardinalcommerce.com 1eafstag.cardinalcommerce.com 1eaf.cardinalcommerce.com centinelapistag.cardinalcommerce.com centinelapi.cardinalcommerce.com *.algolia.net *.algolia.com *.algolianet.com *.amazon.com *.amazon.co.uk *.amazon.co.jp *.amazon.jp *.amazon.it *.amazon.fr *.amazon.es *.amazon.de *.amazonpay.com *.amazonpay.co.uk *.amazonpay.co.jp *.amazonpay.jp *.amazonpay.it *.amazonpay.fr *.amazonpay.es *.amazonpay.de mws.amazonservices.com mws.amazonservices.co.uk mws.amazonservices.co.jp mws.amazonservices.jp mws.amazonservices.it mws.amazonservices.fr mws.amazonservices.es mws.amazonservices.de *.playground.klarnaevt.com www.googletagmanager.com www.google-analytics.com *.criteo.com vars.hotjar.com *.de17a.com *.zdassets.com *.trustpilot.com dnacdn.net *.getsentry.com sparmax.zendesk.com s.sparmax.no tryggehandel.no google.com gtm.sparmax.no *.clarity.ms bat.bing.com wapi.lipscore.com google-analytics.com widget-mediator.zopim.com google.com/recaptcha *.klarnacdn.net *.doubleclick.net *.klarnaevt.com *.klarna.com s.sparmax.se s.sparmax.dk s.maxkjop.no *.skjaraard.no s.skjargaard.no www.facebook.com/tr wss://widget-mediator.zopim.com static.zdassets.com *.klarnauserservices.com *.klarnaservices.com *.trackedlink.net *.trackedweb.net *.dotdigital-pages.com api.comapi.com webchat.dotdigital.com www.paypal.com www.sandbox.paypal.com api.braintreegateway.com api.sandbox.braintreegateway.com client-analytics.braintreegateway.com client-analytics.sandbox.braintreegateway.com *.braintree-api.com *.yotpo.com 'self' 'unsafe-inline'; child-src assets.braintreegateway.com c.paypal.com http: https: blob: 'self' 'unsafe-inline'; default-src 'self' 'unsafe-inline' 'unsafe-eval'; base-uri 'self' 'unsafe-inline';

[Maxwell012]

Сергей Горностаев, я могу что-то с эти сделать или как-то обойти это или вообще просто не получать заголовок?

[Сергей Горностаев]

Maxwell012, нет. В репе aiohttp на этот счëт есть куча issue, одно из которых аж с 2018-го года, и есть майский PR, который пока не принят. Так что только ждать новой версии aiohttp, менять библиотеку или не работать с больными сайтами.

[Maxwell012]

Сергей Горностаев, спасибо за ответ)

[Everything_is_not_so_bad]

https://stackoverflow.com/questions/686217/maximum...