Hi,
I have more than 4 years of experience in the big data field. During this period I have worked on many spark and Hadoop applications.
I have a lot of experience tuning spark applications for performance by changing the spark submit parameters like executor memory, number of cores, overhead memory, broadcast threshold and many more. I also understand the spark resource manager page very well and hence can also identify issues like data skew or some other issue in the code or query which might be the cause of the bottleneck.
I am confident that I will be able to resolve your issue because of my background in this field.
Kindly provide me with further details and I can start working on it right away.
Regards,
Amith