Hello, I'm a data engineer with experience utilising PySpark and AWS Cloud to solve big data problems.
Currently employed at one of the Big4 companies, handling big data business issues.
I have expertise on the following Tools/Technologies:
- Python, Unit test script, Numpy, Pandas
- Apache Spark (PySpark)
- AWS Cloud Services (AWS Lambda, Glue, Step function, s3, EC2 etc)
- DataBricks
- SQL/MySQL
- Git, GitHub, BitBucket
- CI/CD Pipelines
- Big Data Pipelines
- IBM DB2
- PyCharm, VSCode, Jupyter Notebook, MS Excel
Please provide detailed information about your requirement. Thank you!