How statistics are calculated
We count how many offers each candidate received and for what salary. For example, if a Data QA developer with Selenium with a salary of $4,500 received 10 offers, then we would count him 10 times. If there were no offers, then he would not get into the statistics either.
The graph column is the total number of offers. This is not the number of vacancies, but an indicator of the level of demand. The more offers there are, the more companies try to hire such a specialist. 5k+ includes candidates with salaries >= $5,000 and < $5,500.
Median Salary Expectation – the weighted average of the market offer in the selected specialization, that is, the most frequent job offers for the selected specialization received by candidates. We do not count accepted or rejected offers.
Trending Data QA tech & tools in 2024
Data QA
What is Data Quality
A data quality analyst maintains an organisation’s data so that they can have confidence in the accuracy, completeness, consistency, trustworthiness, and availability of their data. DQA teams are in charge of conducting audits, defining the data quality standards, spotting outliers, and fixing the flaws, and play a key role at all stages in the data lifecycle. Without DQA work, strategic plans will fail, operations will go awry, customers will leave, and organisations will face substantial financial losses, as well as a lack of customer trust and potential legal repercussions due to poor-quality data.
This is a job that has changed as much as the hidden infrastructure that transforms data into insight and then powers the apps that we all use. I mean, it’s changed a lot.
Data Correctness/Validation
This is the largest stream of all the tasks. When we talk about data correctness, we should be asking: what does correctness mean to you, for this dataset? Because it would be different for every dataset and every organisation. The commonsense interpretation is that it must be what your end user (or business) wants from the dataset. Or what would be an expected result of the dataset.
We can obtain this just by asking questions, or else reading through the list of requirements. Here are some of the tests we might run, in this stream:
Finding Duplicates — nobody wants this in their data.
– Your data contains unique/distinct values in that column/field. Will the returned value be a unique/distinct value in that column/field?
– Any value that can be found in your data is returned.
Data with KPIs – If data has any columns we can sum, min or max on it’s called a key performance indicator. So basically any models which are mostly numeric/int column. eg: Budget, Revenue, Sales etc. If there is data comparison between two datasets then below tests applies:
– Comparing counts between two datasets — get the difference in count
– Compare the unique/distinct values and counts for columns – find out which values are not present in either of the datasets.
– Compare the KPIs between two datasets and get the percentage difference between them.
– Replace missing values – missing in any one of the datasets with primary or composite primary key. This can be done in a data source that does not have primary key too.
– Perform the metrics by segment for the individual column value — that can help you determine what might be going wrong if the count of values in the Zoopla-side doesn’t match the count on the Rightmove-side or if some of the values are missing.
Data Freshness
This is an easy set. How do we know if the data is fresh?
An obvious indication here is to check if your dataset has a date column, in which case, you just check the max date. Another one is, when the data was pulled into a particular table, all of this can be converted into a very simple automated checks, which we might talk about in a later blog entry.
Data Completeness
This could be an intermediate step in addition to data correctness, but how do we know to get there if the space of answers is complete?
To do this test, check if any column has all values null in it perhaps that’s okay, but most of the time it’s bad news.
Another test would be one-valuedness: whether everywhere on the column all values are the same, probably in some cases that would be a fine result, but probably in other cases that would be something we’d rather look into.
What are Data Quality Tools and How are They Used?
Data quality tools are used to improve, or sometimes automate, many processes required to ensure that data stays fit for analytics, data science, and machine learning. For example, such tools enable teams to evaluate their existing data pipelines, identify bottlenecks in quality, and even automate many remediation steps. Examples of activities relating to guaranteeing data quality include data profiling, data lineage, and data cleansing. Data cleansing, data profiling, measurement, and visualization tools can be used by teams to ‘understand the shape and values of the data assets that have been acquired – and how they are being collected’. These tools will call outliers and mixed formats. In the data analytics pipeline, data profiling acts as a quality control gate. And each of these are data management chores.
Where is Selenium used?
Web Wizardry: Spell-Casting in Automation Tests
- Selenium weaves its magic in automation, making browsers dance to its testing tunes like puppets on strings.
Compatibility Conundrums: Tackling the Browser Babble
- Acting as a universal translator, Selenium harmonizes the babbling browsers, ensuring peace across platform provinces.
Data-Driven Debonair: The Stylish Side of Scripting
- By donning the data-driven tux, Selenium sashays through test scenarios with the elegance of a scripting sommelier.
Continuous Integration Circuses: Juggling Code and Testing Clowns
- In the CI circus, Selenium juggles testing pins with precision, ensuring the code clowns don't trip over their own shoelaces.
Selenium Alternatives
Cypress
An all-in-one testing framework for web applications. Cypress executes tests in a browser directly alongside application code.
cy.visit('https://example.com')
cy.contains('Welcome!')
- Real-time reloads for test development
- Access to browser and device APIs
- Better debugging with DOM snapshots
- Only supports JavaScript
- Runs in a single browser tab
- No support for multi-tabs or multi-windows
Playwright
Node library to automate Chromium, Firefox, and WebKit. Playwright operates across multiple browser contexts.
const { chromium } = require('playwright');
(async () => {
const browser = await chromium.launch();
const page = await browser.newPage();
await page.goto('https://example.com');
})();
- Multi-browser support
- Cross-platform: Linux, Windows, macOS
- Handles modern web features well
- Limited community and resources
- Fairly new, less mature
- Primary focus on headless browsing
TestCafe
An end-to-end, node.js tool. TestCafe automates browser testing and has no need for WebDriver or other testing software.
import { Selector } from 'testcafe';
fixture `Getting Started`
.page `https://example.com`;
test('My first test', async t => {
await t
.typeText('#developer-name', 'John Smith')
.click('#submit-button');
});
- Easy setup with no browsers plugins
- Concurrent test execution
- Integrated test stability mechanisms
- Less performant with heavy usage
- Limited browser support
- Basic built-in assertions
Quick Facts about Selenium
When Browsers Got a Driver's License: Selenium's Birth
In the mystical era of 2004, a smart chap named Jason Huggins, while laboring in the webs of ThoughtWorks, had a lightbulb moment. Watching his tests manually was a no-go, so he created a contraption named Selenium. It's like giving browsers their own driver's license so they could ride the test track without any hands on the steering wheel!
The Flavorful Soup of Selenium Versions
Selenium's been stirring its tech soup with a bunch of flavors. Starting with Selenium RC, which is akin to a telephone game—your code tells the server what to do and the server tells the browser. Fast-forward, and Selenium WebDriver cranks it to 11 by speaking the browser's native language. Then, the shiny Selenium Grid comes along, throwing a party for multiple browsers to join in on the test-athon simultaneously. It's like the browser version of speed dating!
Groundbreaking, Earth-Shaking: Selenium IDE's Rebirth
Rumors had it that the Selenium IDE kicked the bucket in 2017 with the demise of its Firefox extension support. But lo and behold, the phoenix rose from the ashes, with the IDE making a blockbuster comeback! Now it's strutting as a shiny new extension available on the hottest browser runways, Chrome, and Firefox. Like a Hollywood makeover, it went from washed-up to red carpet-ready!
// Here's a taste of Selenium WebDriver in action:
WebDriver driver = new ChromeDriver();
driver.get("https://www.selenium.dev");
WebElement downloadTab = driver.findElement(By.linkText("Downloads"));
downloadTab.click();
What is the difference between Junior, Middle, Senior and Expert Selenium developer?
Seniority Name | Years of Experience | Average Salary (USD/year) | Responsibilities & Activities |
---|---|---|---|
Junior | 0-2 | 50,000-70,000 |
|
Middle | 2-5 | 70,000-90,000 |
|
Senior | 5-10 | 90,000-120,000 |
|
Expert/Team Lead | 10+ | 120,000-150,000+ |
|
Top 10 Selenium Related Tech
Java
Well, if we were stranded on a coding island, Java would be the volleyball we call 'Wilson'! I mean, it's the bedrock for Selenium automation. With its object-oriented approach, you get the Swiss Army knife of programming to handle your tests. You'll write scripts that can run on just about any system, chat up web applications, and not break a sweat about the underlying OS.
// Sample Selenium Java test
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
public class WittyTest {
public static void main(String[] args) {
WebDriver driver = new FirefoxDriver();
driver.get("https://www.example.com");
// More whimsical test code goes here
driver.quit();
}
}Python
Bite into Python if you want to automate web apps like you're snacking on appetizers. The language is clean, efficient, and, dare I say, charmingly snake-like. It coils around any task and just gets it... done. Selenium WebDriver + Python makes writing automation scripts easier than forgetting to mute your mic on a conference call.
# Sample Selenium Python test
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("https://www.example.com")
# Pythonic automation magic happens here
driver.quit()TestNG
Imagine a stage director for your tests: that's TestNG. This framework is no-nonsense when it comes to organizing your Java tests with annotations. From simple unit tests to complex integrated ones, TestNG will have them ordered like ducks in a row. It's like having an assertive personal assistant for your testing suite.
JUnit
The grandparent of Java testing frameworks which refuses to retire. JUnit is everywhere; chances are, if you've tested in Java, you've whispered sweet assertions into JUnit's ear. It's perfect for unit testing, but don't let its age fool you—JUnit can sprint with your Selenium tests like it's running from the Y2K bug.
Maven
A build automation tool that handles your dependencies like an expert juggler spinning plates at a circus. Maven keeps your project-building process smoother than a latte from that over-hipster coffee shop. It uses XML to its advantage, and once set up, you can forget about the hassle of jars and focus on the art of testing.
WebDriverManager
Forget the days of manually downloading drivers for your browsers. WebDriverManager is like the fairy godmother of Selenium tests—it automatically waves its magic wand and poof! Your browser drivers are up-to-date! It's so automatic it could make you believe your computer's possessed... in a good way.
Page Object Model (POM)
Page Object Model isn't a tool or a tech, but a concept for structuring your code. It makes scripts as organized as a librarian with OCD. By modeling your pages into objects, your tests get more reusable, readable, and reliable than grandma's old apple pie recipe—and just as comforting.
Git
Imagine your code had a time machine—that's Git. It tracks your changes with the commitment of a historian, allowing you to branch out with the wild abandon of a soap opera character. Git ensures that when things go sideways, you can rollback like a pro. Bonus: you get to sound cool by saying things like "Just commit it to the repo, bro!"
Docker
Docker swoops in like a superhero, containerizing your Selenium grid and tests into isolated environments. Your applications can now run as consistently as the excuse "It works on my machine." With Docker, you can scale your tests up like they're on a steroid regime—without any of the side effects!
Jenkins
The butler you wished you had for continuous integration. Jenkins runs your Selenium tests with the punctuality of a Swiss watch. It'll integrate with almost anything while wearing a fancy bow tie. Whenever you make a change, Jenkins is there to ensure your code still plays nice, and if it doesn't, it'll let you know—discreetly, like a good butler should.