Saha, Manirupa2015-09-182023-10-262015-09-182023-10-262013ROLL NO. 08612111http://172.17.1.107:4000/handle/123456789/393Supervisor: Amarendra Kumar SarmaModulation instability (MI) is a well known and most ubiquitous and widespread type of instability that appears in most nonlinear systems. In Physics, it appears in many branches of physics such as hydrodynamics, plasma physics, and electrodynamics, low temperature physics, and quite obviously in nonlinear optics. One of the most fascinating and important manifestations of the phenomenon of modulation instability is the solitary waves, commonly called solitons. Solitons are robust, localized traveling waves of permanent form. They are recognized as the modes of a nonlinear system. This thesis is primarily devoted towards study of modulational instability and solitary waves in the following nonlinear optical systems: nonlinear negative index metamaterials, non-Kerr media exhibiting power law nonlinearity and very briefly, silicon wave guides. The recent emergence of the so-called negative refractive index metamaterials (NIM) or simply known as metamaterials (MM) have made nonlinear optics research a very useful and exciting activity. This thesis proposes a new generalized coupled nonlinear field equations for pulse propagation in MM embedded into a Kerr medium. The model successfully recovers previously proposed models by other authors. Moreover, it contains some additional terms like magnetic self-steepening effect connecting both the electric and the magnetic field envelopes. MI analysis is carried out using this new model and is extended later to the case of an MM embedded into a medium with cubicquintic nonlinearity. The generalized nonlinear Schrodinger equation, appropriate to model nonlinear pulse propagation in non-Kerr cubic quintic media is integrated to find the exact solitary wave solutions. The parameter domain restrictions have also been identified in the process of obtaining these solutions. The MI analysis is also carried out in this media and the effect of higher order dispersion and higher order nonlinearity on MI gain spectrum is discussed. The exact dark soliton solutions to the generalized nonlinear Schr५dinger equation with coefficients dependent on the evolution parameters are obtained. Finally, a numerical investigation of solitary wave propagation in a silicon wave guide is reported.enPHYSICSThesis