It is wise to carry a compass and know how to use it and a chart or map and know how to read it. A chart case will keep your chart from turning into a giant spitwad. These days a GPS is quite cheap and very usable, although it does not render the chart and compass obsolete. A thorough knowlege of manual navigation is recommended for anyone planning an extended trip.
Navigation is a fun part of learning to kayak. Get a good book like: Fundamentals of Kayak Navigation, a chart of your favorite paddling waters, a compass and get out there and practice until you can plan a trip and execute it as you planned. Then, after you understand the basics, consider buying a GPS to extend your abilities and knowlege.
For temporary markings on a laminated chart or on a chart case, get a Stabilo Aquarellable pencil. These can write on almost any surface, including many plastics. They also can be used to write on your kayak's deck for quick notes. To erase, just rub with your finger (a little water will help sometimes).