The final and arguably most alien advance usage of に 'ni' is when it is used together with the passive construction to express an adverse condition or action. This has no direct equivalent in English, and if you aren't able to discern the regular passive from the suffering passive construction you may not be able to understand some sentences. In English, one way we could try and express something similar to this is in a roundabout manner such as saying "He up and Xed on me". When に 'ni' is used in this construction, it marks the thing which is causing the negative feeling or the thing which is making you 'suffer'. For example look at the following sentence which uses this pattern.
犬に死なれた 'inu ni shinareta' - My dog up and died on me.
Notice that the verb 死ぬ 'shinu' is in the passive form, 死なれる・死なれた 'shinareru / shinareta'. And note that に 'ni' marks the thing which is the cause of the adverse action. Your dog is the source of your 'suffering' thus you say [dogに 'ni' + passive verb]. Remember this because it's very important. It may be an awkward way of translating it into English but the point is to get you to understand what the grammar and what the Japanese is trying to imply. It is implying a negative action which had an adverse effect on you. This usage is not incredibly common, and I recommend only using it with verbs and situations you've heard it used in before. Trying to create novel sentences with this pattern may turn out sounding awkward to Japanese ears as it's a fairly idiomatic construction.
For your practice, we'll go through two more common examples together.
How would you say "It up and rained on me"? (highlight between the red brackets for the answer). Remember that the verb must use the passive construction, and that which is causing the adverse action must be marked with に 'ni'. Hint: The rain is causing the adverse action. [雨に降られた 'ame ni furareta']
Now how would you say, "She up and cried on me"? Hint: She's not physically crying on you, this is just pointing out that her crying has had an adverse effect on you. [彼女に泣かれた 'kanojo ni nakareta']
As can be seen in these examples, the verb must always be in the passive construction, and whatever is causing you negative feelings or making you 'suffer' must be marked with に 'ni'. In the example 犬に死なれた 'inu ni shinareta' the に 'ni' is marking 犬 'inu' the dog because the dog is the source of your 'suffering'.
Note: This construction is called 「迷惑の受身」 'meiwaku no ukemi' in Japanese. 迷惑 'meiwaku' means to bother or disturb, and 受身 'ukemi' means passive.