Write the part (two, three, or four) and then go back. In my novel I had it divided into two parts and often wrote a story in part two and went to part one to fill in a precursor of why the character "acted" as they did in part two. So it helped in continuity and added to the story.
Write down the idea and basic outline of that specific part and come back to it later. Or start on another part then come back to the first one. Sometimes it helps just to write everything that comes to mind for a solid hour then edit it. Too often people edit as they write which hinders progress.
As everyone else has said, write the parts you have down and come back to fill in the blanks later. Good luck!
I think you're trying to hard just let it come to you