Блог

 import requests
from bs4 import BeautifulSoup
import pandas as pd
start_url="https://www.indeed.co.in/jobs?q=teacheramp;l=India"
page_data=requests.get(start_url)
soup=BeautifulSoup(page_data.content,"html.parser")
Title=[]
Company=[]
Salary=[]
Summary=[]
Location=[]
link_to_apply=[]

for job_tag in soup.find_all("div",class_="jobsearch-SerpJobCard unifiedRow row result"):
    title=job_tag.find("h2",class_="title")
    company=job_tag.find("span",class_="company")
    location=job_tag.find(class_="location accessible-contrast-color-location").text.strip()
    summary=job_tag.find("div",class_="summary")
    link=job_tag.find("a",href=True)
    base_url="https://www.indeed.com"
    final_link=base_url link["href"]

    Title.append(title.text.strip())
    Company.append(company.text.strip())
    Location.append(location)
    Summary.append(summary.text.strip())
    link_to_apply.append(final_link)


data=list(zip(Title,Company,Location,Summary))
p=pd.DataFrame(data,columns=["Title","Company","Location","Summary"])
p.to_csv("new.csv")
pd.set_option("display.max_colwidth",None)
pd.set_option("display.max_rows",None)
pd.set_option("display.max_rows",None)
pd.set_option("display.width",None)
pd.set_option('display.max_columns',None)
  

 the output of the following code

                                                                 Title  
0                                              all Subject Teachernnew   
1                                           Part time Teacher / Trainer   
2   Online Tutor / Teachers - Women Candidates only - Non Metro...nnew   
3                                          Science amp; Maths Teachernnew   
4                                                Primary School Teacher   
5                                                Preschool Teachernnew   
6       Online Tutor / Teachers - Women Candidates only- Across Indi...   
7                                                         Maths Teacher   
8                               wanted faculty for teaching cbse school   
9                                               Primary English Teacher   
10                                                 Training Facilitator   
11                                                         Head Teacher   
12                                                    Math Teachernnew   
13                                                     Teacher for Kids   
14                                            Preschool Teacher - Vizag   

                                                  Company  
0                                               Home Guru   
1                                               Home Guru   
2                 Whitehat Education Technology Pvt. Ltd.   
3                                               Home Guru   
4                       Newdimension International School   
5                                               Home Guru   
6                 Whitehat Education Technology Pvt. Ltd.   
7                            Dheeraj International School   
8   TILFORD SCHOOL,JANGAREDDIGUDEM,W.G.DIST,ANDHRA PRA...   
9                                       Stones2Milestones   
10                                      Stones2Milestones   
11                                           Wunderschool   
12                             GEMS Public School,Patiala   
13                                            ANAR EdTech   
14                                        Koala Preschool   

                                  Location  
0                                   Remote   
1                                   Remote   
2                            Kochi, Kerala   
3                                   Remote   
4                  Bhongir, Andhra Pradesh   
5                      Mumbai, Maharashtra   
6                     Bengaluru, Karnataka   
7                        Pune, Maharashtra   
8   Jangareddi Gudem Bazar, Andhra Pradesh   
9                         Gurgaon, Haryana   
10                        Gurgaon, Haryana   
11                  Chandigarh, Chandigarh   
12                Univ P O Patiala, Punjab   
13                           Kochi, Kerala   
14           Visakhapatnam, Andhra Pradesh   

                                                                                                                                                                Summary  
0    We are recognized leaders in one to one coaching of all subjects amp; Courses, both online and offline.nStudents and Gurus from all across India are invited to get…  
1    We are recognized leaders in one to one coaching of all subjects amp; Courses, both online and offline.nStudents and Gurus from all across India are invited to get…  
2              Teacher, Work from Home, Online Tutor, Teaching, Home Tutor, coding teacher, computer teacher, home teacher, kids teacher.nYou've found your dream job.  
3    We are recognized leaders in one to one coaching of all subjects amp; Courses, both online and offline.nStudents and Gurus from all across India are invited to get…  
4      Proven experience as a teacher.nCollaborate with other teachers, parents and stakeholders and participate in regular meetings.nTotal work: 1 year (Preferred).  
5     We Provide Virtual recorded sessions of Preschool curriculum,.nHave to record Virtual Classes of preschool curriculum.nMust have preschool teaching experience.  
6              Teacher, Work from Home, Online Tutor, Teaching, Home Tutor, coding teacher, computer teacher, home teacher, kids teacher.nYou've found your dream job.  
7                       We are looking for qualified and experienced candidates who can join immediately for the following posts at Dheeraj International School, Pune.  
8                                    WANTED FACULTY FOR TEACHING @CBSE SCHOOL.nSalary not constraint for deserving candidates. *.nSalary: Up to ₹50,000.00 per month.  
9        We are looking to bring into our fold an incredible teacher/facilitator to conduct online classes for children of ages 6-10.nWorking with: Consumer App Team.  
10      We are looking to bring into our fold an incredible teacher/facilitator to conduct online classes for children of ages 6-10.nWorking with: Consumer App Team*.  
11       The Head teacher shall influence the thinking and practice of students, teachers and parents.nPlanning, Documentation , Evaluation, Mentorship, Guidance and…  
12  Welcome to Gems Public School, Patiala...nGEMS is an international education company.nIt is a global advisory and educational management firm, with a network of…  
13     We are* looking for dynamic and vibrant lady tutors with a passion for inspiring our kids to flourish and reach their potentials.nWork at your convenient time.  
14    Our growing Preschool facility is looking for a Young energetic Female Preschool Teacher who can help to create a fun environment and incorporate educational…  
  

я хочу, чтобы вывод находился в соответствующем табличном столбце.Также таблицу необходимо сохранить в ФАЙЛ CSV
вот почему я выбираю pandas. пожалуйста, ответьте быстро и спасибо, ребята
кто-нибудь знает, почему я не получаю правильный фрейм данных? из pandas в качестве вывода
есть ли какой-либо другой способ сделать это?
также в созданном CSV-файле выходные данные поступают правильно, но в python это не так

По большей части текущий код, похоже, работает. Если вы удалите новые строки, это может помочь. Что касается ‘…’, данные на веб-сайте усекаются. Для перехода по каждой ссылке вам понадобится инструмент веб-автоматизации, такой как Selenium. Вы можете использовать tabulate для форматирования выходной таблицы в консоли.

Вот код с удаленными символами новой строки и исправленными отступами.

 import requests
from bs4 import BeautifulSoup
import pandas as pd

start_url="https://www.indeed.co.in/jobs?q=teacheramp;l=India"
page_data=requests.get(start_url)
soup=BeautifulSoup(page_data.content,"html.parser")
Title=[]
Company=[]
Salary=[]
Summary=[]
Location=[]
link_to_apply=[]

for job_tag in soup.find_all("div",class_="jobsearch-SerpJobCard unifiedRow row result"):
    title=job_tag.find("h2",class_="title")
    company=job_tag.find("span",class_="company")
    location=job_tag.find(class_="location accessible-contrast-color-location").text.strip()
    summary=job_tag.find("div",class_="summary")
    link=job_tag.find("a",href=True)
    base_url="https://www.indeed.com"
    final_link=base_url link["href"]

    Title.append(title.text.replace('n'," ").strip())
    Company.append(company.text.replace('n'," ").strip())
    Location.append(location.replace('n'," "))
    Summary.append(summary.text.replace('n'," ").strip())
    link_to_apply.append(final_link.replace('n'," "))
    
data=list(zip(Title,Company,Location,Summary))
p=pd.DataFrame(data,columns=["Title","Company","Location","Summary"])
p.to_csv("new.csv", index=False)
pd.set_option("display.max_colwidth",None)
pd.set_option("display.max_rows",None)
pd.set_option("display.max_rows",None)
pd.set_option("display.width",None)
pd.set_option('display.max_columns',None)
pd.options.display.max_colwidth = None
print(p.to_string(index=False))

# from tabulate import tabulate
# print(tabulate(p, headers='keys', tablefmt='psql'))  # terminal
#p   # Jupyter only, direct output
print(formatdf(p, 25))   # Idle
  

Вывод (Jupyter, усеченный)

Для Idle мне пришлось написать функцию для форматирования данных. Это перенесет данные столбца в фрейм данных на основе максимальной ширины столбца.

 def formatdf(df, mxcolwidth):
    outstr = ''
    for c in df.columns:
        df[c] = df[c].str.wrap(mxcolwidth)  # insert newlines
        
    # get max width for each column
    wdic = {}
    for c in df.columns:
       s = df[c]
       mx = 0
       for r in s:
           for ln in r.split('n'):
               if len(ln) > mx:
                   mx = len(ln)
       wdic[c] = mx  # dictionary, max line length of each column

    # create row divider string 
    rowstr = ''
    for c in df.columns:
        rowstr  = '-' * (wdic[c])
    rowstr  = '-' * (len(df.columns)*3)   '-n'


    outstr  = '| '  # start row line
    # column headers
    for c in df.columns:
        outstr  = c.ljust(wdic[c])   ' | '
    outstr  = 'n'
    outstr  = rowstr
    
    # each row in dataframe
    for ir, r in df.iterrows():
        mxln = 0
        for c in df.columns:  # get mxx lines for this data row
            lncnt = len(r[c].split('n'))
            if lncnt > mxln: mxln = lncnt
            
        for i in range(mxln): # for each line in data cell
            for ic, c in enumerate(df.columns):  # each column
                if ic == 0: outstr  = '| '   # left border of table
                lns = r[c].split('n')  # split, each line of text in data cell
                if i < len(lns):
                    outstr  = lns[i].ljust(wdic[c])   ' | '  # single line of text
                else:
                    outstr  = " ".ljust(wdic[c])   ' | '   # empty line
            outstr = "n"
        outstr  = rowstr  # row divider
              
    return outstr
  

Вывод (бездействующий, усеченный)

 | Title                     | Company                   | Location                 | Summary                   | 
----------------------------------------------------------------------------------------------------------------
| Online Tutor / Teachers - | Whitehat Education        | Srinagar, Jammu and      | Teacher, Work from Home,  | 
| Women Candidates only -   | Technology Pvt. Ltd.      | Kashmir                  | Online Tutor, Teaching,   | 
| Non Metro... new          |                           |                          | Home Tutor, coding        | 
|                           |                           |                          | teacher, computer         | 
|                           |                           |                          | teacher, home teacher,    | 
|                           |                           |                          | kids teacher. You've      | 
|                           |                           |                          | found your dream job.     | 
----------------------------------------------------------------------------------------------------------------
| Post Graduate Teacher and | North Eastern Railway     | Gorakhpur, Uttar Pradesh | North Eastern Railway     | 
| Trained Graduate Teacher  |                           |                          | Recruitment 2020 - Post   | 
| new                       |                           |                          | Graduate Teacher and      | 
|                           |                           |                          | Trained Graduate Teacher  | 
|                           |                           |                          | Vacancies - Apply         | 
|                           |                           |                          | NowNorth Eastern Railway  | 
|                           |                           |                          | Recruitment 2020-21:…     | 
----------------------------------------------------------------------------------------------------------------