Phantomjs Vs Selenium Scraping

Selenium vs PhantomJS | Top 7 Phenomenal Distinction To …

Differences Between Selenium vs PhantomJS
Although both Selenium and PhantomJS are open source options for automation testing, Selenium is much more competent in functional validation for web application testing, and PhantomJS is suitable at most for Smoke testing and Sanity testing. Selenium is mostly used by Testers for all types of testing processes, whereas developers ideally use PhantomJS for unit testing and smoke testing processes before moving the code to the test environment.
Let us study much more about Selenium and PhantomJS in detail:
Software Testing is the most important procedure while producing any Software product. It actually goes hand in hand with the Software Development process. Now, Selenium and PhantomJS are the two equally important and well-known testing frameworks. Jason Huggins initially developed selenium in 2004. Ariya Hidayat released PhantomJS. Selenium has 4 components, such as Selenium IDE, Selenium RC, Selenium Web Driver, and Selenium Grid.
Like the two sides of a coin, these two frameworks also have pros and cons.
Pros and Cons of Selenium vs PhantomJs
The pros and cons are as follows:
Selenium Pros
Large Community Support
Successful Integration with CI tool
Cross-Browser compatibility
Easy integration with an existing testing framework
Different Languages support
Cross-Platform Compatibility
Easy to debug as the real-time activities are visible
Approved WC3 standard web driver
Selenium Cons
Not that suitable for running more than a few browsers on one machine at a time
Slow performance while running multiple test cases
The IDE recording and performance isn’t too powerful and effective
PhantomJs Pros
Capable of executing multiple tests
Being headless, the execution is very fast
No need to integrate with third-party service
Low Resource required
Very useful in Smoke Testing
Supports screen capture
Usage in many open source projects
PhantomJs Cons
Not a good option for web app testers
Lack of Ecosystem
Only serves as a single version of Web Kit
Sometimes requires heavy set up
Head to Head Comparison Between Selenium and PhantomJS
Below are the top 7 comparisons of Selenium and PhantomJS:
Key Differences Between Selenium and PhantomJS
The difference between Selenium and PhantomJS is explained in the below-mentioned points:
Selenium is an open-source automation tool, and it works on real browsers with direct interaction with the UI elements. In the case of PhantomJS, it is a scripted headless browser that is used for automating web page interaction.
Being a headless browser, the interactions are much faster than the real browser. So the performance time is smoother in PhantomJS than in Selenium.
Selenium has one component known as Selenium IDE, which one can use the same IDE for coding to debug. This would not be the case for PhantomJS.
Selenium doesn’t directly support screen capture, and one has to implement add-ins to process the same, whereas PhantomJS supports screen capture.
Selenium supports many browsers (cross browsers) and a good platform for future real browsers to Selenium web driver, whereas PhantomJS is a scripted headless browser and doesn’t work on the real browser (UI) interaction.
Selenium works on a real browser and standalone UI interactions. So, easy to set preconditions for the tests. But this is a bit different in the case of PhantomJS, as it is a headless browser.
Recommended courses:
Comparisons Table Selenium and PhantomJS
Below is the comparison between Selenium and PhantomJS.
BASIS FOR COMPARISON
Selenium
PhantomJS
Definition
Selenium is an open-source automation tool. It supports all major operating systems and browsers like Mozilla, IE and Firefox.
PhantomJS is a headless Web Kit scriptable with a JavaScript API. It is basically an optimal solution for Headless Website Testing, access web pages and comes with the standard DOM API.
Platform
Selenium works on UI and using real browsers. So if any needs to be developed on a specific Javascript engine, it can be reached through Selenium.
PhantomJS uses QtWebKIt, and that means it is not the same javascript engine that Chrome uses. Although it is not a big issue, sometimes real browsers are required instead of headless browsers.
Unit Testing
Unit testing is mainly done by the developers. Now, being worked on real browsers and interacts with UI based on current existing functionality, Selenium is less preferred than PhantomJS for unit testing.
Developers also preferred PantomJS as code can be written by selecting the HTML element while developing the UI. It also becomes an important tool for unit testing.
Smoke Testing
Selenium is used for smoke testing verifying critical changes. Selenium IDE is basically used as a prototyping tool and would not be suitable for creating and maintaining complex test suites. You have to use a web driver and other features for the same.
PhantomJS is a perfect and ideal choice for smoke testing web apps. It basically blends smoothly into the development workflow.
Support
Selenium is an open-source tool and backed by a large community of developers who are continuously working on the creation of plugins that are required for ensuring 100% compatibility of Selenium with all the latest JavaScript front-end frameworks.
PhatomJS is also used and supported by a lot of developers, and they are also continuously working on the new implementations for the same. But Selenium has a much bigger community support.
Simplicity
Selenium is mainly associated with extensive documentation which is simply clear and easy to understand
PhantomJS is a scripted, headless browser used for automation. It can also be learned, but selenium has a broader documentation aspect than PhantomJS.
Advanced Features
Selenium is capable of using different sets of common programming languages like C#, Ruby, Python, and JavaScript.
PhantomJS is mainly used to serve as a single version of Web Kit and so is not a good option for web app testers.
Conclusion
After comparing Selenium and PhantomJS over a range of factors, it can be concluded that the selection of any of the testing framework depends entirely on the features that best fit the project needs, as each one has its own pros and cons. As both Selenium and PhantomJS are perfect in their roles, understanding their unique capabilities would help them decide which one to choose. Thus, based on the project need, type of browsers, time of work, and all the other discussed aspects, any one of these tools should be selected to reach the desired goal.
Recommended Articles
This has been a guide to Differences Between Selenium vs PhantomJS. Here we also discuss the Selenium vs PhantomJS head to head comparison, key differences, along with infographics and a comparison table. You may also look at the following articles to learn more –
Selenium vs Cucumber
Selenium vs QTP
Linux vs Windows 10
Python 3 vs Python 2

Python, PhantomJS says I am not using headless? – Stack Overflow

my code is:
from selenium import webdriver
driver = antomJS(executable_path=’driver/bin/’)
(“)
print(rrent_url)
It seems to run fine but before it runs I always get this error:
UserWarning: Selenium support for PhantomJS has been deprecated, please use headless versions of Chrome or Firefox instead (‘Selenium support for PhantomJS has been deprecated, please use headless
Why am I getting this error? I thought my PhantomJS was headless as it still works and no browser pops-up is this error save to ignore?
Ronak Shah337k18 gold badges108 silver badges164 bronze badges
asked May 18 ’18 at 17:30
5
Selenium considers PhantomJS as deprecated, so you need to us either Chrome or Firefox in headless mode.
Here are the steps to use Chrome in headless mode:
download chrome driver from extract it to a folder
add this folder to your PATH environment variable (if you don’t do it, you will have to use (‘/your/path/to/chromedriver’) in the code below instead of ())
Then use it like this:
# prepare the option for the chrome driver
options = romeOptions()
d_argument(‘headless’)
# start chrome browser
browser = (chrome_options=options)
(”)
()
More on how to use ChromeDriver
For the other options: here (also here and here)
answered Dec 15 ’18 at 10:54
MagTunMagTun4, 7633 gold badges46 silver badges86 bronze badges
4
In Selenium 3. 8. 1 PhantomJS marked as deprecated webdriver and recommend us to use either Chrome or Firefox in headless mode.
answered Nov 25 ’18 at 6:25
You could use this:
browser = (‘chromedriver_path/chromedriver’)
answered Aug 6 at 16:21
jarh1992jarh19923564 silver badges6 bronze badges
Found an alternative you can add d_argument(‘headless’) to chrome
answered May 18 ’18 at 17:46
OgdenOgden4671 gold badge7 silver badges20 bronze badges
0
Not the answer you’re looking for? Browse other questions tagged python selenium phantomjs or ask your own question.

PhantomJS is dead, long live headless browsers – Puzzle ITC

Many frontend projects still rely on PhantomJS to run JavaScript tests. As of spring 2017, PhantomJS is not supported anymore and you should migrate your project to an alternative environment. Here is what you can do.
In April 2017, Vitaly Slobodin announced, that he’s stepping down as a developer and maintainer of PhantomJS, the headless WebKit browser. This is mainly due to the fact that Google introduced Headless Chrome with Chrome 59. And since version 55, Firefox also provides a headless mode.
There are several reasons to favor headless Chrome/Firefox over PhantomJS:
They are real browsers with a broad feature support (PhantomJS uses a very old version of WebKit – and in the meanwhile Chrome switched to Blink anyway)
They are faster and more stable (PhantomJS has a lot of open issues)
They use less memory
They can be started non-headless, which allows easier debugging
No more goofy PhantomJS binary installation with NPM
In the next sections I’m going to suggest a few alternatives to a PhantomJS setup and elaborate on their advantages and disadvantages.
Alternative 1: Don’t use a browser at all
It may sound a little irritating at first, but you should seriously think about not using a browser at all to execute your JavaScript unit tests. Many React projects are already doing this with Jest for example, where the DOM is abstracted with jsdom (a pure-JavaScript implementation of a subset of the DOM and HTML standards). It is possible to use Jest in Angular projects too.
The advantage is that these tests run way faster and can be executed completely within Node. This also means no special setup on the CI server is needed. The downside is that they are not executed in a real browser and you have to mock browser APIs. Additionally, if you have end-to-end tests, you are going to need a real browser setup anyway.
Alternative 2: Use headless Chrome (or Firefox)
In a more conventional setup with Karma, the switch from PhantomJS to Chrome is quite easy. Instead of the karma-phantomjs-launcher, you install the karma-chrome-launcher and configure Karma accordingly in your
module. exports = function (config) {
({
plugins: [
require(‘karma-chrome-launcher’), ],
browsers: [‘Chrome’], });};
This will open a Chrome window and execute the tests within the browser. Chances are, you are already using this setup for local debugging.
The karma-chrome-launcher also supports a headless preset which makes working with Headless Chrome dead simple. You only have to change the preset:
browsers: [‘ChromeHeadless’],
The launcher assumes that the Chrome binary is available on the system (if in an exotic location, you can provide a CHROME_BIN environment variable). The launcher supports Chromium as well with the Chromium and ChromiumHeadless presets (for the latter, make sure you have version >= 2. 2. 0).
So far so good, but what about running the tests on a CI server? For Travis, there is a Chrome addon that can be included. And Jenkins? You probably don’t want to install Chrome/Chromium (and it’s dependencies) on every slave. Furthermore, you cannot just install Chrome/Chromium via NPM1 or download and unpack it2 since you’d still need to install all the libraries it is dynamically linked to.
1 yes, there are some shady packages you shouldn’t trust
2 although puppeteer does exactly this
Alternative 3: Use a cloud service like Sauce Labs
With the karma-sauce-launcher, running tests with various browsers is easy (locally as well as on the CI server). You configure custom launchers for each browser type and toss in the connection credentials as environment variables. Et voilà.
Sauce Labs is a paid service.
Alternative 4: Launch Chrome in a Docker container
A very naive approch is to run Chrome in a Docker container. For this, we create a Dockerfile that installs Chromium and exposes its remote debugging port:
FROM debian:stable-slim
MAINTAINER Mathis Hofer <>
RUN apt-get update -y && apt-get install -y -q chromium
EXPOSE 9222
ENTRYPOINT [
“/usr/bin/chromium”,
“–no-sandbox”,
“–no-default-browser-check”,
“–no-first-run”,
“–disable-default-apps”,
“–disable-popup-blocking”,
“–disable-translate”,
“–disable-background-timer-throttling”,
“–headless”,
“–disable-gpu”,
“–remote-debugging-address=0. 0. 0”,
“–remote-debugging-port=9222”] You can then build this image and start the container. I’ve created a script that does this, taking a URL as argument:
#! /usr/bin/env bash
TESTS_URL=$1
SCRIPT_DIR=”$( cd “$( dirname “$0″)” && pwd)”
docker build -t chromium-headless $SCRIPT_DIR
if [ -n “`docker ps -a | grep chromium-headless`”]; then
docker rm –force chromium-headless
fi
docker run
–rm
-p=127. 1:9222:9222
–net=host
–name chromium-headless
chromium-headless
$TESTS_URL
By using the karma-script-launcher, we can configure Karma to use this script to start Chromium. It then executes the tests with Chromium running in a Docker container:
require(‘karma-script-launcher’), ],
browsers: [‘. /config/docker/chromium-headless/’], });};
While it is pretty promising to be able to use the same image with the exact same browser version locally, there are some issues with this method:
Your test setup has to know about the Docker setup and has to be adapted accordingly
On the CI server, Docker has to be installed and it must be allowed to do a docker build and docker run within the environment of the job.
How do you ensure the image is rebuilt regularly to update to new browser versions?
How do you handle concurrent test jobs (container name, debugging port)?
How do you clean up containers?
Alternative 5: Dynamic Jenkins slave with the Docker Slave plugin
So when adopting Docker, why not go all the way and manage the whole Jenkins slave with Docker? This is possible with the Docker Slaves plugin. The plugin enables you to setup build agents using Docker containers by placing a Dockerfile in your source repository and set up the job to use it (any image is supported). You can also define side containers (for the database etc. ) similar to docker-compose.
The advantage of this option is that your frontend test/build setup has to know nothing about Docker.
Alternative 6: Dynamic Jenkins slave on OpenShift
When working with a Kubernetes/OpenShift cluster, the Jenkins Kubernetes plugin is an interesting option.
OpenShift offers a bunch of preconfigured images that work with the Kubernetes plugin (e. g. openshift/jenkins-slave-base-centos7). You can use them as base image to build an image containing Chromium. Then create an OpenShift build from your Dockerfile with the oc new-build command.
Furthermore, a new pod template has to be created (Manage Jenkins > Cloud > Kubernetes), where the URL to the Docker image(-stream) is configured. The pod configuration options are described here.
Now create a Jenkins (Multi-)Pipeline project for your Git repository and configure the label of the template you defined above in the project’s Jenkinsfile:
pipeline {
agent { label ‘name-of-your-pod-template’}
//…
stages {
stage(‘Frontend Unit Tests’) {
steps {
sh “””#! /bin/bash +x
source ${HOME}/
nvm install 8
yarn
ng test –browsers=ChromiumHeadlessNoSandbox
–single-run=true
–code-coverage
“””}}}}
What about artifacts? They have to be archived to survive a pod shutdown. Jenkins Plugins like JUnit or Cobertura already pull the concerned files out of the container and copy them onto the Jenkins master. Any other artifacts can be archived with archiveArtifact.
As you may have noticed, the custom ChromiumHeadlessNoSandbox preset is used in this example. This is due to the inability of Chrome’s sandboxing feature to work in a Docker container as-is. For our testing context we can live with disabling the sandbox with a custom laucher in
customLaunchers: {
ChromiumHeadlessNoSandbox: {
base: ‘ChromiumHeadless’,
flags: [‘–no-sandbox’]}},
Let’s run the job! When analyzing the output, we can observe that the tests are executed in a container using headless Chromium:
Running on openshiftv3slave-headlesschrome-x1z5c in /home/jenkins/workspace/chrome-headless-test_master-GBIHLYMXJN4A7BETBYE5KU2AUCXVX7ORN3AWON5KZBLUWR54PJYA……
13:34:32 20 12 2017 12:34:31. 496:INFO [karma]: Karma v1. 7. 1 server started at 13:34:32 20 12 2017 12:34:31. 498:INFO [launcher]: Launching browser ChromiumHeadlessNoSandbox with unlimited concurrency
13:34:32 20 12 2017 12:34:31. 503:INFO [launcher]: Starting browser ChromiumHeadless
13:34:38 20 12 2017 12:34:37. 572:INFO [HeadlessChrome 0. 0 (Linux 0. 0)]: Connected on socket EjirM7jznbVydLR3AAAA with id 22863249
13:34:40 HeadlessChrome 0. 0): Executed 0 of 3 SUCCESS (0 secs / 0 secs)
13:34:40 HeadlessChrome 0. 0): Executed 1 of 3 SUCCESS (0 secs / 0. 143 secs)
13:34:40 HeadlessChrome 0. 0): Executed 2 of 3 SUCCESS (0 secs / 0. 189 secs)
13:34:40 HeadlessChrome 0. 0): Executed 3 of 3 SUCCESS (0 secs / 0. 235 secs)
13:34:40 HeadlessChrome 0. 0): Executed 3 of 3 SUCCESS (0. 272 secs / 0. 235 secs)…
Finished: SUCCESS
Last but not least the browser has be kept up-to-date. This can be achieved by periodically rebuilding the image with another Jenkins job like this:
agent any
options {
disableConcurrentBuilds()}
triggers {
cron(‘@weekly’)}
stage(‘Start build’) {
script {
withCredentials([… ]) {
openshiftBuild apiURL: ‘your-openshift-environment’, authToken:… }}}}}}
Conclusion
PhantomJS is a thing of the past, but the good news is there are compelling alternatives with the headless modes of Chrome and Firefox. Although the overall complexity may rise, especially when Docker comes into play.
Please contact us if you have questions regarding a similar scenario.
What are your experiences on the journey replacing PhantomJS?
Image credit: „Valparaíso Puerto“ by Mathis Hofer, 2010, CC BY-SA 3

Frequently Asked Questions about phantomjs vs selenium scraping

Does Selenium support PhantomJS?

Selenium considers PhantomJS as deprecated, so you need to us either Chrome or Firefox in headless mode.Nov 25, 2018

Can I still use PhantomJS?

As of spring 2017, PhantomJS is not supported anymore and you should migrate your project to an alternative environment. Here is what you can do. In April 2017, Vitaly Slobodin announced, that he’s stepping down as a developer and maintainer of PhantomJS, the headless WebKit browser.Feb 12, 2018

Is Selenium better than Scrapy?

In short, If the job is a very simple project, then Selenium can be your choice. If you want a more powerful and flexible web crawler, or you indeed have some experience in programming, then Scrapy is definitely the winner here.Jan 2, 2021