When you make an Skill Roll, the rules or the GM determines how difficult the roll could be or if it is being used against a target for a Contesting Skill Roll.
As Player Characters level up, they get Skill Points which can remove Disadvantage Stack from the Main Roll of non-Innate Skill Roll.