При get запросе через requests основная часть полученного html кода сайта пропадает.
header = {
'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36',
'Content-Type':'application/x-www-form-urlencoded',
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Language':'ru-ru,ru;q=0.8,en-us;q=0.5,en;q=0.3',
'Accept-Encoding':'gzip, deflate',
'Connection':'keep-alive',
'DNT':'1'
}
reg_url = 'https://foxford.ru/user/registration'
session = requests.session()
data = session.get(reg_url, headers=header).content
page = lxml.html.fromstring(data)
print(data)
Полученный код:
<!DOCTYPE html><html><head><meta charset="utf-8" /><meta content="!" name="fragment" /><meta content="IE=edge" http-equiv="X-UA-Compatible" />
<script></script> Бла-бла-бла скрипты
<!-- Google Tag Manager -->
<noscript><iframe src="//www.googletagmanager.com/ns.html?id=GTM-NDCLX6"
height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript>
<script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=
'//www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
})(window,document,'script','dataLayer','GTM-NDCLX6');</script>
<!-- End Google Tag Manager -->
<script></script
<b><i><div id="BaseApp-react-component-40e0b07f-cd07-49bc-b5ed-f15351755f0c"></div></i></b>
<script src="https://s.foxford.ngcdn.ru/js-libs/mathjax@2.7.7/MathJax.js?config=TeX-AMS_SVG" id="MathJax-script" async="async"></script><script src="https://a.foxford.ngcdn.ru/assets/webpack/js/runtime.deaad9a6.js" defer="defer"></script>
<script src="https://a.foxford.ngcdn.ru/assets/webpack/js/vendors.87c41ba3.chunk.js" defer="defer"></script>
<script src="https://a.foxford.ngcdn.ru/assets/webpack/js/common.ui.08fe203f.chunk.js" defer="defer"></script>
<script src="https://a.foxford.ngcdn.ru/assets/webpack/js/lms.06bd8984.chunk.js" defer="defer"></script></body></html>
А именно
<div id="BaseApp-react-component-40e0b07f-cd07-49bc-b5ed-f15351755f0c"></div>
абсолютной пустой. Хотя должен содержать форму для регистрации.
