Adding a fixed skeg to the aft end of a kayak noticeably changes two handling attributes of a kayak. It makes it harder to turn and it reduces weathercocking. If you ever paddle a kayak that weathercocks (or leecocks) in a brisk crosswind you will quickly discover it is unmanageable and exhausting. So if your kayak weathercocks (I am sure it does if it is oversteering), this is what I would do (and have done to mine).

-Round the forward edge of the coaming at the rear of the cockpit as much a possible.
-Move the seat aft as mush as is comfortable to layback for a roll. This should leave about 3 to 4 inches clear between your back and the coaming when seated.
-Test paddle in crosswind to confirm the kayak weathercocks.
-Add a fixed skeg or extension of the keel down.
-Test paddle in crosswind. If it still weathercocks add more skeg or aft keel and test paddle. Repeat if necessary until your kayak leecocks (turns down wind.).
-Cut and file some skeg/keel off, seal with epoxy and test paddle in a crosswind. Repeat until the kayak is wind neutral(does not weathercock or leecock). Then apply final finish.

This will be the best you can do by adding a fixed skeg without doing major alteration to your kayak (such as moving your cockpit aft or adding rocker at the bow).