In this paper, we introduce a new class of Hermite poly-Genocchi polynomials and we give some identities of those polynomials related to the Stirling numbers of the second kind. The concepts of poly-Bernoulli numbers $B^{(k)}_n(a,b)$, poly-Bernoulli polynomials $B^{(k)}_n(x,a,b)$ of Jolany et al, Hermite-Bernoulli polynomials $HB_n(x,y)$ of Dattoli et al and $HB^{(\alpha)}_n(x,y)$ of Pathan et al are generalized to the one $HG^{(k)}_n(x,y)$. Some implicit summation formulae and general symmetry identities are derived by using different analytical means and applying generating functions. These results extend some known summations and identities of Hermite poly-Genocchi numbers and polynomials.