Not gaining muscle and I'm working hard?

Seriously, I work my legs and upper body each week at the gym. I always walk out with my muscles ripped up and buldging huge and I eat lots of food and chug lots of powder every few hours. And then a few days later I'm like the size I was before I went to the gym. I'm not gaining muscle I do everything right I put in all this effort just to waist my time. I'm also an ectomorph is this why I struggle to gain any muscle?