From Dictionary.com - Specific jargon usage deleted:
sys·tem /ˈsɪstəm/ [sis-tuhm] –noun
1. an assemblage or combination of things or parts forming a complex or unitary whole: a mountain system; a railroad system.
2. any assemblage or set of correlated members: a system of currency; a system of shorthand characters.
3. an ordered and comprehensive assemblage of facts, principles, doctrines, or the like in a particular field of knowledge or thought: a system of philosophy.
4. a coordinated body of methods or a scheme or plan of procedure; organizational scheme: a system of government.
5. any formulated, regular, or special method or plan of procedure: a system of marking, numbering, or measuring; a winning system at bridge.
6. due method or orderly manner of arrangement or procedure: There is no system in his work.
12. ( sometimes initial capital letter ) the prevailing structure or organization of society, business, or politics or of society in general; establishment (usually prec. by the ): to work within the system instead of trying to change it.
All of these definitions work to help comprehend what a roleplaying game system is. Many games use system as a synonym for resolution mechanic. This is wrong. A mechanic is a component of a system. in all the meanings of the word above, System refers to a coordinated assemblage of working parts. The resolution mechanic is one of those parts.
Some components of a roleplaying game system are systems themselves - most magic in games is set up as a system rather than a monolithic entity. Character generation is another typically systematized component. These do not have to be systems in themselves, but typically this is so.
Here are the typical component parts of roleplaying games:
Character Generation - creating a character ex nihilo. Point allocation, random rolling, lifepaths either random or determined, choosing descriptors, templates either simple or overlain, and variations from the norm. These techniques can be used either individually or in combinations to create characters. For example in StarCluster 3 I am using all of the above in one form or another.
Character Advancement - the growth of the generated character in ability from experience in play. gain abilities in levels, purchase single abilities with XP, gain in ability by accumulating items or components, gain in ability purely over time, undergo special training. These techniques are used to model the character's increasing mastery over his environment.
Powers/Magic/PSI - the ability of the charater to use exceptions to influence the setting. Vancian fire-and-forget, using Magic Points, burning attributes or lifepoints, ad-hoc casting, rituals, spell components, preparation, refreshing resources, spell lists, combinng spells/powers, combining power from several characters. These techniques codify exceptions - the ways the character can influence the setting which violate the physics of the setting.
Resolution Mechanics - the way the system adjudicates conflict. Task resolution, scene resolution, conflict resolution, dice pools, bell curves, straight curves, exploding dice, roll and keep, roll over, roll under, Chance of Success/Failuer, Quality of Success/Failure, unified mechanics, diversified mechanics, karma, fallout, bennies, resource allocation, playing cards. These techniques are used for resolution of conflicts - task resolution is after all conflict resolution by accumulated tasks. Who wins, who loses, and what is the price paid are answered by these techniques. These things are sometimes operated through the character, but generally they are operated by the player directly.
Initiative - the method of determining when the various characters act within the structure of the conflict. Roll high, roll low, stat determined, resource allocation, rounds both short and long, ticks, cycles, maneuvers, simulaneity. These techniques structure time in conflict to determine when actions happen, and may be rolled into the Resolution mechanic, or may be entirely separate.
Resources - the things the character owns or controls which can be used to aid in mastering the environement or aid in conflict. Equipment, tools, magic items, technological items, animals, vehicles, shelter, weapons, slaves/servants, money, favors both owed and granted. These resources are controlled by the player through the medium of the character, and exist in the game setting separately from the character.
Abilities - the qualities inherent in the character which can be used to aid in mastering the environement or aid in conflict. Skills, level abilities, masteries, attributes, stats, power reserves, point pools, refresh rates, Life points, Hit points, life levels, traits, edges, specializations. These are the direct means by which the character directly manipulates the world, and thus the ways the player manipulates the game world through the character.
By combining the various compenents together - coordinating them, they become a system. The system is the structure and coordination of these components, not just the components themselves. Every system has its own flavor, and like anything with flavor, some people are particular while others are broadly accepting, and tastes vary. That's there are a so many systems out there. :D