Web Automation with Selenium

Selenium is a powerful Python library used for web automation, testing, and scraping dynamic websites. It allows you to:
Automate form submissions
Interact with web elements (buttons, inputs, etc.)
Scrape JavaScript-rendered content
Perform UI testing

---

2. Installing Selenium

Before using Selenium, install it via pip:

bashCopyEditpip install selenium

You also need a WebDriver (Chrome, Firefox, Edge, etc.).

Download WebDriver:

🔹 Chrome – Download ChromeDriver
🔹 Firefox – Download GeckoDriver

Place the WebDriver in your project folder or set the system path.

---

3. Setting Up Selenium

Launch Chrome Browser

from selenium import webdriver

driver = webdriver.Chrome()  # Launch Chrome browser
driver.get("https://www.google.com")  # Open Google

print(driver.title)  # Print the page title
driver.quit()  # Close the browser

Now you have successfully opened a website!

---

4. Locating Web Elements

To interact with a webpage, identify elements using Selenium locators:

• find_element(By.ID, "element_id")
• find_element(By.NAME, "element_name")
• find_element(By.CLASS_NAME, "class_name")
• find_element(By.TAG_NAME, "tag_name")
• find_element(By.XPATH, "xpath_expression")
• find_element(By.CSS_SELECTOR, "css_selector")

Example: Locate a Search Box in Google

from selenium.webdriver.common.by import By

driver = webdriver.Chrome()
driver.get("https://www.google.com")

search_box = driver.find_element(By.NAME, "q")  # Locate search box
search_box.send_keys("Selenium Web Automation")  # Enter text
search_box.submit()  # Press Enter

print(driver.title)
driver.quit()

This script searches for “Selenium Web Automation” in Google!

---

5. Clicking Buttons and Links

Clicking a Button

button = driver.find_element(By.XPATH, "//input[@value='Google Search']")
button.click()

Clicking a Link by Text

link = driver.find_element(By.LINK_TEXT, "Images")
link.click()

Now you can click elements dynamically!

---

6. Handling Dropdowns, Checkboxes, and Radio Buttons

Select from a Dropdown

from selenium.webdriver.support.ui import Select

dropdown = Select(driver.find_element(By.ID, "dropdown_id"))
dropdown.select_by_visible_text("Option 1")

Click a Checkbox

checkbox = driver.find_element(By.ID, "checkbox_id")
checkbox.click()

🔹 Select a Radio Button

radio_button = driver.find_element(By.NAME, "radio_name")
radio_button.click()

Now you can interact with form elements!

---

7. Handling Alerts & Pop-ups

Accept an Alert

alert = driver.switch_to.alert
alert.accept()

Dismiss an Alert

alert.dismiss()

Now you can handle JavaScript alerts!

---

8. Taking Screenshots

driver.save_screenshot("screenshot.png")

This saves a screenshot of the current webpage!

---

9. Handling Multiple Tabs and Windows

Switch Between Tabs

driver.switch_to.window(driver.window_handles[1])  # Switch to the second tab

Now you can automate multi-tab browsing!

---

10. Automating Login Forms

driver.get("https://example.com/login")

driver.find_element(By.NAME, "username").send_keys("your_username")
driver.find_element(By.NAME, "password").send_keys("your_password")
driver.find_element(By.NAME, "login").click()

Now you can automate website logins!

---

11. Scrolling the Web Page

Scroll Down

pythonCopyEditdriver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

Scroll to an Element

element = driver.find_element(By.ID, "footer")
driver.execute_script("arguments[0].scrollIntoView();", element)

Now you can automate scrolling!

---

12. Scraping JavaScript-Rendered Content

Extract Page Content

page_content = driver.page_source
print(page_content)

Now you can extract dynamically loaded content!

---

13. Automating Web Tasks (Filling Forms, Extracting Data)

Example: Extract Google Search Results

driver.get("https://www.google.com")
search_box = driver.find_element(By.NAME, "q")
search_box.send_keys("Python Selenium")
search_box.submit()

results = driver.find_elements(By.CSS_SELECTOR, "h3")
for result in results:
    print(result.text)

driver.quit()

Now you can extract search results!

---

14. Running Selenium Headless (Without GUI)

Run in Background Mode

options = webdriver.ChromeOptions()
options.add_argument("--headless")
driver = webdriver.Chrome(options=options)

Now Selenium runs invisibly!

---

15. Scheduling Automated Web Tasks

Use schedule to run tasks at specific times:

import schedule
import time

def task():
    driver.get("https://example.com")
    print("Task executed")

schedule.every().day.at("08:00").do(task)

while True:
    schedule.run_pending()
    time.sleep(60)

Now tasks run automatically!

---

16. Deploying Selenium Automation on a Server

For 24/7 automation, deploy on:
PythonAnywhere (Cloud-based automation)
AWS Lambda (Serverless execution)
Docker (Containerized automation)