Очистка Python — невозможно получить требуемые данные из Flipkart

#python #html #beautifulsoup

#python #selenium #beautifulsoup

Вопрос:

Я пытался очистить отзывы клиентов с веб-сайта Flipkart. Ниже приведена ссылка. Ниже приведен мой код для очистки, но он всегда возвращает пустой список.

 >>> from bs4 import BeautifulSoup
>>> import requests

>>> r = requests.get('https://www.flipkart.com/samsung-galaxy-j5-6-new-2016-edition-white-16-gb/product-reviews/itmegmrnzqjcpfg9?pid=MOBEG4XWJG7F9A6Z')
>>> soup = BeautifulSoup(r.content, 'lxml') # Tried with 'html.parser' also
>>> soup.find_all('div', '_3DCdKt')
[]
>>> soup.find_all('div', {'class': '_3DCdKt'})
[]
>>> soup.find_all('div', {'class': 'row _3wYu6I _3BRC7L'})
[]
>>> soup.find_all('div', {'class': '_1GRhLX hFPo14'})
[]
  

Итак, я попытался получить весь раздел, но я получал только следующее:

 >>> soup.find_all('div', {'class': 'col-9-12'})
[<div class="col-9-12" data-reactid="96"><div class="row _2_xtR5" data-reactid="97"></div><div class="row _3wYu6I _1KVtzT" data-reactid="98"></div></div>]
  

Я не получал другое содержимое. Итак, затем я попробовал с selenium, даже тогда он возвращался None . Ниже приведен мой код selenium:

 >>> driver = webdriver.Firefox()
>>> driver.get('https://www.flipkart.com/samsung-galaxy-j5-6-new-2016-edition-white-16-gb/product-reviews/itmegmrnzqjcpfg9?pid=MOBEG4XWJG7F9A6Z')
>>> a = driver.find_elements_by_class_name("_3DCdKt")
>>> len(a)
10
>>> for i in a:
...    print i.get_attribute('value')
...
None
None
None
None
None
None
None
None
None
None
  

В чем может быть проблема? Я делаю какие-либо ошибки в коде. Пожалуйста, помогите. Я новичок в Python.

Ответ №1:

Обзоры и т. Д. Заполняются с помощью reactjs, данные извлекаются с помощью ajax-запроса, который вы можете имитировать с помощью запросов:

 import requests

data = {"productId": "MOBEG4XWJG7F9A6Z", # end of url pid=MOBEG4XWJG7F9A6Z
        "count": "15",
        "ratings": "ALL",
        "reviewerType:ALL"
        "sortOrder": "MOST_HELPFUL"}



headers = ({"x-user-agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.92 Safari/537.36 FKUA/website/41/website/Desktop"})
data = requests.get("https://www.flipkart.com/api/3/product/reviews", params=data, headers=headers).json()
print(data)
  

То, что вы хотите, это получить доступ data["RESPONSE"]["data"] к списку dicts:

 for dct in data["RESPONSE"]["data"]
    print(dct)
  

Что даст вам:

 {u'action': None, u'fixed': False, u'value': {u'rating': 5, u'text': u'Thanks to Flipkart who deliver it me with in 5 days nGood Phone With Metal Body nAnd Best front Camera With FlashnBest for night Selfie nI Take more than 30 pic in night mode with front flash ngood smartphone  gold color is also superebnbest ever smartphone under 15k by samsungnGood BatterynGood Camera Front with Flash and Rear Also Superb', u'reportAbuse': {u'action': {u'originalUrl': None, u'params': {u'vote': u'ABUSE', u'reviewId': u'be37810e-20fe-4417-9d88-2709288cf2ba', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 0, u'type': u'VoteValue'}, u'tracking': None}, u'totalCount': 285, u'downvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'DOWN', u'reviewId': u'be37810e-20fe-4417-9d88-2709288cf2ba', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 74, u'type': u'VoteValue'}, u'tracking': None}, u'id': u'be37810e-20fe-4417-9d88-2709288cf2ba', u'author': u'Happy Thakur', u'url': u'/reviews/be37810e-20fe-4417-9d88-2709288cf2ba', u'upvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'UP', u'reviewId': u'be37810e-20fe-4417-9d88-2709288cf2ba', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 211, u'type': u'VoteValue'}, u'tracking': None}, u'helpfulCount': 211, u'created': u'16 May, 2016', u'certifiedBuyer': True, u'title': u'Best Smartphone by Samsung', u'type': u'ProductReviewValue'}, u'tracking': None}
{u'action': None, u'fixed': False, u'value': {u'rating': 5, u'text': u"Updated Review on 02-August after 3 months of usage:nWhat I liked most:nLook : 100/100 - Very good looking phone. Gold color and the finishing is super coolnSize : 100/100 - 5.2 Inch is neither big nor small. I can still operate with one hand.. nBattery : 100/100 - 3100 mAH is outstanding. 3G is always ON when i am out of home and Wi-Fi is always ON in home. I am charging mobile only once in every 36 hours. I use Whatsapp, instagram and Browsing mostly. nDisplay : 90/100 - Not so bright and sharp as S series phones, but a real deal for the price. Impressed again. My only worry is about it is not having a Gorilla scratch proof glass. I may need to use tempered glass.nTouch : 95/100 - So smooth and I dont see any lags as of now.nCamera : 90/100 - Photos are good and can capture fast, but again not as great as S series phones. but at this price I believe this phone outclasses all other competitors in camera department. nnOne last thing is about the SAMSUNG brand and its service center coverage, which is again awesome. nOverall I am completely satisfied with the phone and this phone reached my expectations. nWhat I disliked:nEarphone jack at the bottom.. I feel uncomfortable when chatting and listening to songs at same timenLow speaker volume, not a big deal though for me, As i don't use loudspeaker for songs mostly", u'reportAbuse': {u'action': {u'originalUrl': None, u'params': {u'vote': u'ABUSE', u'reviewId': u'e786669a-024b-4ef0-b70c-1e4fcf5fe5ff', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 0, u'type': u'VoteValue'}, u'tracking': None}, u'totalCount': 272, u'downvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'DOWN', u'reviewId': u'e786669a-024b-4ef0-b70c-1e4fcf5fe5ff', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 87, u'type': u'VoteValue'}, u'tracking': None}, u'id': u'e786669a-024b-4ef0-b70c-1e4fcf5fe5ff', u'author': u'Naresh Kareti', u'url': u'/reviews/e786669a-024b-4ef0-b70c-1e4fcf5fe5ff', u'upvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'UP', u'reviewId': u'e786669a-024b-4ef0-b70c-1e4fcf5fe5ff', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 185, u'type': u'VoteValue'}, u'tracking': None}, u'helpfulCount': 185, u'created': u'13 May, 2016', u'certifiedBuyer': True, u'title': u'Absolute Stunner and Impressive', u'type': u'ProductReviewValue'}, u'tracking': None}
{u'action': None, u'fixed': False, u'value': {u'rating': 3, u'text': u'Hi,nnI got this phone from Flipkart on Friday and here is my 3 days review.nnPros:n * Beautiful designn * Very handy, easy to handlen * Battery backup is greatn * Back camera is goodn * No heating issuesn nCons:n * If we are charging, it will not show any light or any notification whether it is charging or not. We need to on the screen and check whether it is charging or not. So every time we need to turn it on and see whether it is charging or not.n* Camera issue: Once you take the picture and then press the back button it is taking some time to come back to camera mode.n* If you turn on the flash and take pic with back camera it is taking some time to capture the picture. With out Flash it is taking very fast.n* Volume is very low. Not enough for a medium sized room.n* Ear phones are not good especially for me. nnnWill post my feedback after using it another 15 days.nnThanks', u'reportAbuse': {u'action': {u'originalUrl': None, u'params': {u'vote': u'ABUSE', u'reviewId': u'9cbcd27c-a8ad-4793-978a-5903cd086252', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 0, u'type': u'VoteValue'}, u'tracking': None}, u'totalCount': 212, u'downvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'DOWN', u'reviewId': u'9cbcd27c-a8ad-4793-978a-5903cd086252', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 67, u'type': u'VoteValue'}, u'tracking': None}, u'id': u'9cbcd27c-a8ad-4793-978a-5903cd086252', u'author': u'ileep ', u'url': u'/reviews/9cbcd27c-a8ad-4793-978a-5903cd086252', u'upvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'UP', u'reviewId': u'9cbcd27c-a8ad-4793-978a-5903cd086252', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 145, u'type': u'VoteValue'}, u'tracking': None}, u'helpfulCount': 145, u'created': u'16 May, 2016', u'certifiedBuyer': True, u'title': u'Good looking phone with some drawbacks', u'type': u'ProductReviewValue'}, u'tracking': None}
{u'action': None, u'fixed': False, u'value': {u'rating': 5, u'text': u'Super Amoled Display..2 GB RAM with Latest Android Marshmallow OS only for 13K....its difficult to get Samsung Phone with 2 GB ram in such a low price Range...used for 15 days....Going Smooth....Awesome Earphone Quality.....selfie and back Camera Good.....Battery last for more than a day with Continous usage or will go for two days....Free Microsoft apps and Much More...', u'reportAbuse': {u'action': {u'originalUrl': None, u'params': {u'vote': u'ABUSE', u'reviewId': u'1546ed16-5945-4257-9f2d-0d86db7ed92e', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 0, u'type': u'VoteValue'}, u'tracking': None}, u'totalCount': 34, u'downvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'DOWN', u'reviewId': u'1546ed16-5945-4257-9f2d-0d86db7ed92e', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 9, u'type': u'VoteValue'}, u'tracking': None}, u'id': u'1546ed16-5945-4257-9f2d-0d86db7ed92e', u'author': u'Prashant Dias', u'url': u'/reviews/1546ed16-5945-4257-9f2d-0d86db7ed92e', u'upvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'UP', u'reviewId': u'1546ed16-5945-4257-9f2d-0d86db7ed92e', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 25, u'type': u'VoteValue'}, u'tracking': None}, u'helpfulCount': 25, u'created': u'7 Sep, 2016', u'certifiedBuyer': True, u'title': u'Brilliant Phone Compared to Money', u'type': u'ProductReviewValue'}, u'tracking': None}
{u'action': None, u'fixed': False, u'value': {u'rating': 5, u'text': u"Nice.battery backup it's good", u'reportAbuse': {u'action': {u'originalUrl': None, u'params': {u'vote': u'ABUSE', u'reviewId': u'a9f2f6a0-2272-4187-bd37-48eb8a0a85c9', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 0, u'type': u'VoteValue'}, u'tracking': None}, u'totalCount': 5, u'downvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'DOWN', u'reviewId': u'a9f2f6a0-2272-4187-bd37-48eb8a0a85c9', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 0, u'type': u'VoteValue'}, u'tracking': None}, u'id': u'a9f2f6a0-2272-4187-bd37-48eb8a0a85c9', u'author': u'Flipkart Customer', u'url': u'/reviews/a9f2f6a0-2272-4187-bd37-48eb8a0a85c9', u'upvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'UP', u'reviewId': u'a9f2f6a0-2272-4187-bd37-48eb8a0a85c9', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 5, u'type': u'VoteValue'}, u'tracking': None}, u'helpfulCount': 5, u'created': u'17 Aug, 2016', u'certifiedBuyer': True, u'title': u"It's very good", u'type': u'ProductReviewValue'}, u'tracking': None}
{u'action': None, u'fixed': False, u'value': {u'rating': 5, u'text': u'This Phone is awesome..Must Buy', u'reportAbuse': {u'action': {u'originalUrl': None, u'params': {u'vote': u'ABUSE', u'reviewId': u'cf8cf2c8-1f79-4d56-a4cd-e641ffb3551b', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 0, u'type': u'VoteValue'}, u'tracking': None}, u'totalCount': 5, u'downvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'DOWN', u'reviewId': u'cf8cf2c8-1f79-4d56-a4cd-e641ffb3551b', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 0, u'type': u'VoteValue'}, u'tracking': None}, u'id': u'cf8cf2c8-1f79-4d56-a4cd-e641ffb3551b', u'author': u'Durvank Aregekar', u'url': u'/reviews/cf8cf2c8-1f79-4d56-a4cd-e641ffb3551b', u'upvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'UP', u'reviewId': u'cf8cf2c8-1f79-4d56-a4cd-e641ffb3551b', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 5, u'type': u'VoteValue'}, u'tracking': None}, u'helpfulCount': 5, u'created': u'10 Aug, 2016', u'certifiedBuyer': True, u'title': u'Must Buy', u'type': u'ProductReviewValue'}, u'tracking': None}
{u'action': None, u'fixed': False, u'value': {u'rating': 5, u'text': u'It is a good phone', u'reportAbuse': {u'action': {u'originalUrl': None, u'params': {u'vote': u'ABUSE', u'reviewId': u'ce31beb5-5c8f-4a2d-be7d-aba416592df2', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 0, u'type': u'VoteValue'}, u'tracking': None}, u'totalCount': 5, u'downvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'DOWN', u'reviewId': u'ce31beb5-5c8f-4a2d-be7d-aba416592df2', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 0, u'type': u'VoteValue'}, u'tracking': None}, u'id': u'ce31beb5-5c8f-4a2d-be7d-aba416592df2', u'author': u'Sourabh Jain', u'url': u'/reviews/ce31beb5-5c8f-4a2d-be7d-aba416592df2', u'upvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'UP', u'reviewId': u'ce31beb5-5c8f-4a2d-be7d-aba416592df2', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 5, u'type': u'VoteValue'}, u'tracking': None}, u'helpfulCount': 5, u'created': u'9 Aug, 2016', u'certifiedBuyer': True, u'title': u'Good phone', u'type': u'ProductReviewValue'}, u'tracking': None}
{u'action': None, u'fixed': False, u'value': {u'rating': 5, u'text': u'delivery is in time but my phone is heat will data is on plz check', u'reportAbuse': {u'action': {u'originalUrl': None, u'params': {u'vote': u'ABUSE', u'reviewId': u'1fcf5a13-edef-4b16-8372-8732819c143c', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 0, u'type': u'VoteValue'}, u'tracking': None}, u'totalCount': 9, u'downvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'DOWN', u'reviewId': u'1fcf5a13-edef-4b16-8372-8732819c143c', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 1, u'type': u'VoteValue'}, u'tracking': None}, u'id': u'1fcf5a13-edef-4b16-8372-8732819c143c', u'author': u'Santhoaha m n santhu', u'url': u'/reviews/1fcf5a13-edef-4b16-8372-8732819c143c', u'upvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'UP', u'reviewId': u'1fcf5a13-edef-4b16-8372-8732819c143c', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 8, u'type': u'VoteValue'}, u'tracking': None}, u'helpfulCount': 8, u'created': u'12 Aug, 2016', u'certifiedBuyer': True, u'title': u'very good', u'type': u'ProductReviewValue'}, u'tracking': None}
{u'action': None, u'fixed': False, u'value': {u'rating': 4, u'text': u'Good Product by SamsungnnThe things from this phone isn       1. Marshmellow v6.0n       2. Front flash with 5mb camera not so goodn       3. Its designn       4. Primary Camera is not so good with 13mb led flashn       5. Battery life is also not so goodn       6. Its size is correct in its designn       7. Supports OTGn       9. Only 2GB RAMn      10. 16GB Internal storage but only 11GB is availiablen      11. 4G supportsn      12. Ultra power saving moden      13. S bike moden      14. Speaker volume is not so goodn      15. 3G supportsn      16. Ultra data savingn      17. No auto brightnessn      18. 2G supportsn      19. Top performance n      20. Good phone at the price 14000n      *********************', u'reportAbuse': {u'action': {u'originalUrl': None, u'params': {u'vote': u'ABUSE', u'reviewId': u'958efa75-1b67-4872-9f71-b18035fafe6a', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 0, u'type': u'VoteValue'}, u'tracking': None}, u'totalCount': 20, u'downvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'DOWN', u'reviewId': u'958efa75-1b67-4872-9f71-b18035fafe6a', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 5, u'type': u'VoteValue'}, u'tracking': None}, u'id': u'958efa75-1b67-4872-9f71-b18035fafe6a', u'author': u'Vaishnav ', u'url': u'/reviews/958efa75-1b67-4872-9f71-b18035fafe6a', u'upvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'UP', u'reviewId': u'958efa75-1b67-4872-9f71-b18035fafe6a', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 15, u'type': u'VoteValue'}, u'tracking': None}, u'helpfulCount': 15, u'created': u'17 May, 2016', u'certifiedBuyer': True, u'title': u'By Expert     -Vaishnav VJ', u'type': u'ProductReviewValue'}, u'tracking': None}
{u'action': None, u'fixed': False, u'value': {u'rating': 4, u'text': u'Very nice device', u'reportAbuse': {u'action': {u'originalUrl': None, u'params': {u'vote': u'ABUSE', u'reviewId': u'c7177dfb-39c2-4c0b-8bbd-288f96757c3a', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 0, u'type': u'VoteValue'}, u'tracking': None}, u'totalCount': 4, u'downvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'DOWN', u'reviewId': u'c7177dfb-39c2-4c0b-8bbd-288f96757c3a', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 0, u'type': u'VoteValue'}, u'tracking': None}, u'id': u'c7177dfb-39c2-4c0b-8bbd-288f96757c3a', u'author': u'Flipkart Customer', u'url': u'/reviews/c7177dfb-39c2-4c0b-8bbd-288f96757c3a', u'upvote': {u'action': {u'originalUrl': None, u'params': {u'vote': u'UP', u'reviewId': u'c7177dfb-39c2-4c0b-8bbd-288f96757c3a', u'reviewDomain': u'PRODUCT'}, u'loginType': u'LEGACY_LOGIN', u'url': None, u'fallback': None, u'type': u'REVIEW_VOTE', u'omnitureData': None, u'screenType': None, u'tracking': {}}, u'fixed': False, u'value': {u'count': 4, u'type': u'VoteValue'}, u'tracking': None}, u'helpfulCount': 4, u'created': u'8 Sep, 2016', u'certifiedBuyer': True, u'title': u'Good quality product', u'type': u'ProductReviewValue'}, u'tracking': None}
  

Требуется x-user-agent , без него вы получите 403. Вы можете поиграть с параметрами, чтобы увидеть разные результаты, я оставлю это на ваше усмотрение.

Комментарии:

1. Я попытался изменить страницу, я указал параметр "page": "2" в данных, но я получал ту же информацию, что и раньше. Я попытался получить URL-адрес пути, используя data.request.path_url и вставил URL-адрес в браузер, но он возвращает ошибку. Не могли бы вы помочь мне с получением отзывов с других страниц?

Ответ №2:

Мне показалось, что это сработало для меня

 from selenium import webdriver
from bs4 import BeautifulSoup
driver = webdriver.Firefox()
driver.get('https://www.flipkart.com/samsung-galaxy-j5-6-new-2016-edition-white-16-gb/product-reviews/itmegmrnzqjcpfg9?pid=MOBEG4XWJG7F9A6Z')
html = driver.page_source
soup = BeautifulSoup(html, 'lxml')
for entry in soup.find_all('div', '_3DCdKt'):
    print(entry.text)
  

Возможно, измените entry.text на just entry , в зависимости от ваших потребностей.

Вывод был:

 5★Best Smartphone by SamsungThanks to Flipkart who deliver it me with in 5 days Good Phone With Metal Body And Best front Camera With FlashBest for night Selfie I Take more than 30 pic in night mode with front flash good smartphone  gold color is also superebbest ever smartphone under 15k by samsungGood BatteryGood Camera Front with Flash and Rear Also SuperbREAD MOREHappy ThakurCertified Buyer16 May, 201621776PermalinkReport Abuse
5★Must BuyThis Phone is awesome..Must BuyREAD MOREDurvank AregekarCertified Buyer10 Aug, 201680PermalinkReport Abuse
3★Good looking phone with some drawbacksHi,I got this phone from Flipkart on Friday and here is my 3 days review.Pros: * Beautiful design * Very handy, easy to handle * Battery backup is great * Back camera is good * No heating issues Cons: * If we are charging, it will not show any light or any notification whether it is charging or not. We need to on the screen and check whether it is charging or not. So every time we need to turn it on and see whether it is charging or not.* Camera issue: Once you take the picture a...READ MOREileep Certified Buyer16 May, 201615167PermalinkReport Abuse
5★Absolute Stunner and ImpressiveUpdated Review on 02-August after 3 months of usage:What I liked most:Look : 100/100 - Very good looking phone. Gold color and the finishing is super coolSize : 100/100 - 5.2 Inch is neither big nor small. I can still operate with one hand.. Battery : 100/100 - 3100 mAH is outstanding. 3G is always ON when i am out of home and Wi-Fi is always ON in home. I am charging mobile only once in every 36 hours. I use Whatsapp, instagram and Browsing mostly. Display : 90/100 - Not so bright and s...READ MORENaresh KaretiCertified Buyer13 May, 201618787PermalinkReport Abuse
5★It's very goodNice.battery backup it's goodREAD MOREFlipkart CustomerCertified Buyer17 Aug, 201660PermalinkReport Abuse
5★Good phoneIt is a good phoneREAD MORESourabh JainCertified Buyer9 Aug, 201660PermalinkReport Abuse
5★Brilliant Phone Compared to MoneySuper Amoled Display..2 GB RAM with Latest Android Marshmallow OS only for 13K....its difficult to get Samsung Phone with 2 GB ram in such a low price Range...used for 15 days....Going Smooth....Awesome Earphone Quality.....selfie and back Camera Good.....Battery last for more than a day with Continous usage or will go for two days....Free Microsoft apps and Much More...READ MOREPrashant DiasCertified Buyer7 Sep, 2016269PermalinkReport Abuse
5★very gooddelivery is in time but my phone is heat will data is on plz checkREAD MORESanthoaha m n santhuCertified Buyer12 Aug, 201681PermalinkReport Abuse
4★By Expert     -Vaishnav VJGood Product by SamsungThe things from this phone is       1. Marshmellow v6.0       2. Front flash with 5mb camera not so good       3. Its design       4. Primary Camera is not so good with 13mb led flash       5. Battery life is also not so good       6. Its size is correct in its design       7. Supports OTG       9. Only 2GB RAM      10. 16GB Internal storage but only 11GB is availiable      11. 4G supports      12. Ultra power saving mode      13. S bike mode      14. Spe...READ MOREVaishnav Certified Buyer17 May, 2016155PermalinkReport Abuse
4★Look is great but camera is a bit slow.I have been using this phone for the past one month. Look is great and also has metal body. So far I have only noticed a bit slowness in the camera while capturing a photo and switching between your photo gallery and camera. Other than that, phone works fine (good battery back-up) - need to see how long it lastsREAD MOREFlipkart CustomerCertified Buyer17 Aug, 201692PermalinkReport Abuse
  

Selenium работает медленно, поэтому, если вам это не нужно (см. Ответ Падрайка Каннингема), я бы не стал его использовать.

Комментарии:

1. Я получаю только пустой список следующим образом: >>> soup.find_all('div', '_3DCdKt') [] Если возможно, не могли бы вы показать результат?