One thing that's been pretty heavily implied in previous posts but not directly stated is the importance of misdirection. Setting up a twist isn't just about foreshadowing something and leaving everything in the dark for a reveal, it's about compelling the audience to first establish their own reasonable conclusion, and then directly subverting them.
To put it in a metaphor, it's the difference between gradually turning on the lights in a dark room (a mystery but not a twist), and letting the your friend turn on the lights in his room without him knowing that you replaced actually all the bulbs with blacklights (twist).
The reason why the twist in the Sixth Sense is compelling is because the audience is, based on information given to them on the movie, given reasonable grounds to firmly believe the premise that Bruce Willis is alive, and the audience has no overtly obvious reason to believe that this premise is untrue, even though it is hinted at.
That said, misdirection is a thing that is very easy to mishandle and can cause complete clunkiness in narrative flow. As stated before, it is very difficult to have a twist work when it so blatantly undermines the premise of the plot a la The Village, as opposed to one that uses the established premise in an unexpected but plausible way a la The Sixth Sense.
Let's use another metaphor.
The premise is "Bob eats a fruit". The fruit is revealed to be a banana. This is no twist, because the audience has no reason to firmly believe that the fruit in question isn't a banana.
The premise is "Bob eats an orange-coloured fruit," The fruit is revealed to be a strawberry. This is a terrible use of misdirection because the reveal makes no sense in relation to the conclusion that the audience will logically reach (i.e. that Bob is eating an orange).
The premise is "Bob eats an orange-coloured fruit." The fruit is revealed to be a tomato. This is a "twist" because the audience reaches a logical, fairly straightforward conclusion (that the fruit is an orange) and have no reason to believe that it isn't true (it is described as orange-coloured and as a fruit), but the conclusion (tomatoes are fruits and can be orange coloured) is reasonable within the established premise.
Silly metaphors aside, essentially when you foreshadow, it has to have two sides. One for misdirection, and the other for the actual twist.
To put it in a metaphor, it's the difference between gradually turning on the lights in a dark room (a mystery but not a twist), and letting the your friend turn on the lights in his room without him knowing that you replaced actually all the bulbs with blacklights (twist).
The reason why the twist in the Sixth Sense is compelling is because the audience is, based on information given to them on the movie, given reasonable grounds to firmly believe the premise that Bruce Willis is alive, and the audience has no overtly obvious reason to believe that this premise is untrue, even though it is hinted at.
That said, misdirection is a thing that is very easy to mishandle and can cause complete clunkiness in narrative flow. As stated before, it is very difficult to have a twist work when it so blatantly undermines the premise of the plot a la The Village, as opposed to one that uses the established premise in an unexpected but plausible way a la The Sixth Sense.
Let's use another metaphor.
The premise is "Bob eats a fruit". The fruit is revealed to be a banana. This is no twist, because the audience has no reason to firmly believe that the fruit in question isn't a banana.
The premise is "Bob eats an orange-coloured fruit," The fruit is revealed to be a strawberry. This is a terrible use of misdirection because the reveal makes no sense in relation to the conclusion that the audience will logically reach (i.e. that Bob is eating an orange).
The premise is "Bob eats an orange-coloured fruit." The fruit is revealed to be a tomato. This is a "twist" because the audience reaches a logical, fairly straightforward conclusion (that the fruit is an orange) and have no reason to believe that it isn't true (it is described as orange-coloured and as a fruit), but the conclusion (tomatoes are fruits and can be orange coloured) is reasonable within the established premise.
Silly metaphors aside, essentially when you foreshadow, it has to have two sides. One for misdirection, and the other for the actual twist.